Designing a RESTful API
API is an acronym for “Application Program Interface”. A WEB API can be used in many kinds of applications and acts as a kind of mediator between two applications. If you are creating a website, or a Single Page Application (SPA), or a mobile application, or when you need to communicate to a service on the server-side, or even if one application needs to communicate with another application (for example: a client to a server), an API can be used for that. It’s also possible to use API with Artificial Intelligence, Machine Learning, Cloud, Microservices, Big data, and many other technologies. In order to design a good API, there are some principles that can be followed. In this article, I explain some of these principles.
Before start to talk about REST itself, I’m going to explain a bit about the HTTP Protocol. The HTTP Protocol is used to communicate with the server through requests and responses. For example, in the case of a WEB application, the UI will make a request to the service in the back-end, the service will handle this request and will return a response. This is an example of how HTTP works :
HTTP is a protocol which allows the fetching of resources, such as HTML documents. It is the foundation of any data exchange on the Web and it is a client-server protocol, which means requests are initiated by the recipient, usually the Web browser. Clients and servers communicate by exchanging individual messages (as opposed to a stream of data). The messages sent by the client, usually a Web browser, are called requests and the messages sent by the server as an answer are called responses.
An HTTP Request contains three parts:
- Verb: Which is the action itself, for example, Get, Post, Put, and so on
- Headers: Which contains the information about the request; It’s possible to add some useful information like a cookie, a JWT token, and others.
- Content: Where the information that needs to be written is added. For example, when a new customer is being inserted, the customer data will be in the request message.
An HTTP Response (which is a piece of data in text) contains three parts:
- Status code — Each status code means something, for example: 200 means “ok”, 201 means “created”, 404 means “Bad Request”, and so on
- Headers — Can contain some useful information
- Content — Any content that the server sends back. For example, when a list of customers is requested, this list will come in the response message.
HTTP defines a set of request methods to indicate the desired action to be performed for a given resource. Although they can also be nouns, these request methods are sometimes referred to as HTTP verbs.
The basics verbs are:
- GET — which request a representation of a specific resource. It is used only to return data.
- POST — which is used to submit an entity to a specific resource. It is used to create a new resource.
- PUT — which is used to replace all the properties of a resource. It us used to update a resource.
- PATCH — which is used to apply partial modifications (this is used when it’s not necessary to update all the resource, but only part of it).
- DELETE — which is used to delete a resource.
There are other verbs, but those are the most common verbs that are used. With them it’s possible to perform the CRUD (Create, Update, Read, Delete) operations.
The status code is returned in the response of the API. It’s a number that represents what kind of success or failure happened. When the client makes a request to an API, it will return a status code from the server after the request is made. Some of the status code are:
- 200 — Ok: This status means that the request was successful.
- 201 — Created: This status means that the request has succeeded and a new resource has been created as a result. This is usually returned after a POST request.
- 204 — No Content: We can return this status when we do not want to return anything.
- 400 — BadRequest: This is a generic status for error. It means that the server could not understand the request due to invalid syntax.
- 401 — Unauthorized: This status means that the client is not authenticated, and he should authenticate to do the request.
- 403 — Forbidden: This status means that the client is authenticated but he does not have permission to do what he is trying to do. Unlike 401, the client’s identity is known to the server.
- 404 — Not found: This status means that the server could not find the requested resource.
- 500 — Internal Server Error: This is a generic answer from REST API, it means that the server has encountered a situation it doesn’t know how to handle.
- 503 — Service Unavailable: This status means that the server is not ready to handle the request.
If you want to know more about the other codes, you can read in the Mozilla documentation by clicking here.
REST — REpresentational State Transfer
The term “REST” it’s an acronym for “Representational State Transfer”. It is an architectural style for distributed hypermedia systems and was first presented by Roy Fielding in 2000 in his famous dissertation (you can read this document by clicking here).
REST, or REpresentational State Transfer, is an architectural style for providing standards between computer systems on the web, making it easier for systems to communicate with each other.
REST has six guiding constraints which must be satisfied if an interface needs to be referred to as RESTfull. These are the principles:
- Client-server — Separation of concerns is the principle behind the client-server constraints. By separating the user interface concerns from the data storage concerns, we improve the portability of the user interface across multiple platforms and improve scalability by simplifying the server components. Perhaps most significant to the Web, however, is that the separation allows the components to evolve independently, thus supporting the Internet-scale requirement of multiple organizational domains.
- Stateless — The communication must be stateless in nature. Each request from the client to server must contain all of the information necessary to understand the request, and cannot take advantage of any stored context on the server. Session state is therefore kept entirely on the client.
- Cacheable — Cache constraints require that the data within a response to a request be implicitly or explicitly labelled as cacheable or non-cacheable. If a response is cacheable, then a client cache is given the right to reuse that response data for later, equivalent requests. All the requests you made should support cacheability.
- Uniform interface — By applying the software engineering principle of generality to the component interface, the overall system architecture is simplified and the visibility of interactions is improved. Implementations are decoupled from the services they provide, which encourages independent evolvability.
- Layered system — The layered system style allows an architecture to be composed of hierarchical layers by constraining component behaviour such that each component cannot “see” beyond the immediate layer with which they are interacting.
- Code on demand (optional) — REST allows client functionality to be extended by downloading and executing code in the form of applets or scripts. This simplifies clients by reducing the number of features required to be pre-implemented. Allowing features to be downloaded after deployment improves system extensibility. However, it also reduces visibility, and thus is only an optional constraint within REST.
REST (Representational State Transfer) refers to a group of software architecture design constraints that bring about efficient, reliable and scalable distributed systems.
The basic idea of REST is that a resource, e.g. a document, is transferred via well-recognized, language-agnostic, and reliably standardized client/server interactions. Services are deemed RESTful when they adhere to these constraints.
HTTP APIs in general are sometimes colloquially referred to as RESTful APIs, RESTful services, or REST services, although they don’t necessarily adhere to all REST constraints. Beginners can assume a REST API means an HTTP service that can be called using standard web libraries and tools.
A REST API is an application program interface (API) that uses HTTP requests to get or manipulate data. In the REST architecture, the client sends a request to get or modify resources, and the server sends a response to those requests. To do those requests is used HTTP Verbs.
The term “Restful” refers to a pragmatic approach to using REST, which means that the Web API apply the REST principles. It’s not rare to see an API that is not completely RESTful, and the reason is that is a bit difficult to apply all the principles all the time, so it’s always good to analyse each situation and don't be so restricted on that. Make sure that you are building the best for the application itself.
The key abstraction of information in REST is a resource. Any information that can be named can be a resource: a document or image, a temporal service, a collection of other resources, a non-virtual object (e.g. a person), and so on. REST uses a resource identifier to identify the particular resource involved in an interaction between components.
Each URI points to a resource, and resource means things that represent the objects in your system, for example: products, customers, books, and so on. You can also think of resources as Domain Models/Entities. They are things that you want to Get, Insert, Update and Delete.
The fundamental concept in any RESTful API is the resource. A resource is an object with a type, associated data, relationships to other resources, and a set of methods that operate on it. It is similar to an object instance in an object-oriented programming language, with the important difference that only a few standard methods are defined for the resource (corresponding to the standard HTTP GET, POST, PUT and DELETE methods), while an object instance typically has many methods.
Example of RESTfull API
In this section, I will show an example of a REST API and some principles that are considered when thinking about RESTfull. For this example, let’s consider a scenario where it’s necessary to handle books. We will have an endpoint to create, get all books, get a single book, update and delete a book.
Conventions in URIs
In REST, the URIs are the path to the Resources. To demonstrate some examples, I’m running a Web API locally. In this case, the base URI is:
https://localhost:5001/. After the base URI with the address, the object/resource is specified, which in this case is the books, so this is how the URI is:
Imagine a scenario where you have a CRUD operation to handle with books. This is how it would be using the RESTful convention:
GET /api/books (to get all the books)
GET /api/books/1 (to get a single book searching by id 1)
PUT /api/books/1 (to update a book with id 1)
DELETE /api/books/1 (to delete a book with id 1)
POST /api/books (to create a book)
Note that each endpoint contains an HTTP Verb, following by “
/api” and “
/books”. If we were working with “customer” instead of “book”, it would be: “
/api/customers/”. This is a convention. Also for each operation, it is used a specific HTTP VERB (eg.: GET, POST, DELETE, or other).
Nouns are preferable when we are working with RESTful APIs. So instead of using verbs in the design of the APIs, we should use nouns. For example, instead of having something like
/deleteBook, it will be:
/books, and the HTTP Verb will specify which kind of operation will be. In general it will also be pluralized (
books instead of
book), unless in cases that will handle with a single item, for example
book/title, in the case when a single book will be returned.
In the URI, also some unique identifier should be used because each URI must point to a specific resource. In the list of endpoints above, the id of the book is being used in some operations (Get, Put and Delete), but it is not mandatory to use the id, could also be some other unique identifier.
It’s also possible to use query strings for non-resource properties. It’s generally used elements as formating, sorting, searching, and others. They are query strings because they are not part of the URI itself, they are about optional arguments to those URIs or to those resources. So for example, we can have something like:
/books?page=2, or others.
In order to demonstrate the API requests, I’m running an API locally using Swagger (if you are using .NET 5, Swagger comes automatically configured when a Web API project is created; if you want to know more about swagger you can access Swagger’s website by clicking here). It’s also possible to use tools like Postman to make the requests to an API. This is the example with Swagger (OpenAPI):
The effect of a specific request should depend on whether the resource is a collection or an individual item. The following table summarizes the common conventions adopted by most RESTful implementations using the e-commerce example. Not all of these requests might be implemented — it depends on the specific scenario:
A REST API must be idempotent. Idempotent means “operation that can be applied multiple times without changing the result”. This means that the operations that are executed by the API must always result in the same side effect.
In the case of GET, PUT, PATCH and DELETE, it should always do the same thing. The GET should always return the same data (unless, of course, is something was changed in the system), PUT and PATCH should always do the same change if necessary, and DELETE should delete the item or return an error. So For example. in case that you execute a PUT multiple time, it should not fail in the second or third request because nothing was changed, but it should always work. An exception for idempotent is POST, which is never idempotent. Every time you POST to an API, it will always return a new object/resource.
So doesn’t matter if you make the same request multiple times, the result must always be the same, unless of course, in cases where some data was changed by another request between the calls.
Caching is another requirement when we think about RESTfull API. Of course that not all the APIs need to have caching, but in order to scale up, caching is something that you should consider.
“Caching would be useless if did not significantly improve performance. The goal of caching in HTTP/1.1 is to eliminate the need to send requests in many cases, and to eliminate the need to send full responses in many other cases.” (HTTP standard)
Caching is a technique that stores a copy of a given resource and serves it back when requested. When a web cache has a requested resource in its store, it intercepts the request and returns its copy instead of re-downloading from the originating server. This achieves several goals: it eases the load of the server that doesn’t need to serve all clients itself, and it improves performance by being closer to the client, i.e., it takes less time to transmit the resource back. For a web site, it is a major component in achieving high performance. On the other side, it has to be configured properly as not all resources stay identical forever: it is important to cache a resource only until it changes, not longer.
It’s possible to have server-side caching (and this is good), which means that for example if two or more clients ask for the same book, you cached it on the server in order to return it faster. But the caching in a REST API means making use of HTTP for caching mechanism. For example, when you make an HTTP request to ask for something and include the last version that the data were given, the response should be a 304 — Not Modified. So in this case the client will ask the server if he has the latest version without the server having to find it and then send it back so you can compare it.
The performance of web sites and applications can be significantly improved by reusing previously fetched resources. Web caches reduce latency and network traffic and thus lessen the time needed to display a representation of a resource. By making use of HTTP caching, Web sites become more responsive.
If-MatchHTTP request header makes the request conditional. For
HEADmethods, the server will send back the requested resource only if it matches one of the listed
PUTand other non-safe methods, it will only upload the resource in this case.
Another way of doing HTTP caching is using a header called If-Match on the request. So in this case it will be sent an identifier to specify the version it has on the server, and then when the client tries to update (PUT) for example, in a way of doing concurrency, it will check if the version that the client sends is the same version on the server, and if positive the changes will be applied, otherwise, it will return a 412 — Precondition Failed, because the Header becomes a precondition. These are two examples of how the If-Match can be used:
If-Match: "67ab43", "54ed21", "7892dd"
Entity Tags (ETags)
A nice way to handle this caching is by using Entity Tags (Etags). It supports strong and week caching. Strong caching is used to support things that will live for a big period of time, for example, if you need to cache something for three weeks. Week caching is for things that will have a very short-lived time.
ETagHTTP response header is an identifier for a specific version of a resource. It lets caches be more efficient and save bandwidth, as a web server does not need to resend a full response if the content has not changed. Additionally, etags help prevent simultaneous updates of a resource from overwriting each other ("mid-air collisions").
These ETags will be returned in the response with some unique identifier that represents the version of the resource on the server, and it will be returned in the header. For example:
In the case where you need to return a week type, you start with
W/ and then use the same format in an ETAG, this way it’s possible to indicate to the developer how strong this caching support:
This unique identifier will be returned in the header and can also be used for tracking purpose.
Avoiding mid-air collisions
Using ETag and If-Match can be handy to detect mid-air edit collisions. For example, when something is updated, a unique identifier will be added into an Etag header in the response:
When someone update, the request will contain the
If-Match header containing the
ETagvalues to check:
If the unique identifier doesn’t match, it means that the resource has been edited in-between and a 412 — Precondition Failed error will be thrown.
Supporting Concurrency — RESTfull
A way to support concurrency is having a component that generates ETags. This is how it works:
For demonstration purpose, let’s consider a scenario where we have two clients: Client 1, and Client 2. This is what you can see in the image above:
1 — Client 1 sends a request to get a Book, and the ETag “123456789” was returned.
2 — In this meantime, Client 2 also:
2.1 — send a request to get a Book and the ETag “123456789” is returned
2.2 — and then Client 2 send a PUT request to update the book passing the ETag “123456789” in the If-Match header, and then the API checks this header, compares with the ETag saved for that response, and if they match (in this case, they do), the update will be applied, and at this moment a new ETag will be generated for the response, the ETag “987654321”.
3 — After that, Client 1 sends the update with an If-Match header containing “1234567989”, which is the ETag that Client 1 currently has, and what will happen now is that it will reach the API, and the API will check that this ETag does not match with the most current ETag for that resource, so the API will return a 412 Precondition failed, and the update from Client 1 will not be applied because he was working on an older version of the book (he should first get a fresh copy of the book to work on).
Caching of unchanged resources
Another typical use of the
ETag header is to cache resources that are unchanged. If a user visits a given URL again (that has an
ETag set), and it is stale (which is too old to be considered usable), the client will send the value of its
ETag in an
If-None-Match header field:
The server compares the client’s
ETag (sent with
If-None-Match) with the
ETag for its current version of the resource, and if both values match (that is, the resource has not changed), the server sends back a 304 — Not Modified status, without a body, which tells the client that the cached version of the response is still good to use (fresh).
TIP: When using Postman, by default, a configuration is set to send a no-cache header. So in order to test caching it’s possible to disable this option in the settings.
Versioning the API
Versioning the API is important in order to prevent breaking changes to the clients. For example, imagine that one or more client are using your API, and you made some changes in the API which affect the way how the data is structured, and once this new change is published, it will not be possible for the clients to use the API anymore unless they also update on their side. So in order to avoid a situation like that, we can work with versioning.
So for example, the first version of the API will be v1.0, and the second will be v2.0. When version 2 is published, it will still be possible to use version 1 for some period of time, and you can notify your clients saying that in a certain period of time the version 1 will not be available anymore and they need to update to version 2. After this period of time, you can deactivate version 1 of the API. This way when you update your API, even if you update something that can cause a break changes, this will not directly affect who is already using it, and you will give your clients some time in order to them update their application with the latest version of your API.
This is also handy in situations where you have many clients, and one of them need a specific feature as soon as possible, but this feature can cause some break changes. In this case, you can create a new version, and the client who needs this new feature can already use it, and the other clients will have some time to update.
Breaking changes should always result in a change to the major version number for an API or content response type. Breaking changes include:
- a change in the format of the response data for one or more calls
- a change in the request or response type (i.e. changing an integer to a float)
- removing any part of the API
If the API is only used by your own team and your own application, maybe create versioning for the API is too overkill, but in cases where the API is being used by internal or external customers, versioning the API it’s very important.
Strategies to versioning the API
REST doesn’t provide any specific versioning guidelines but some of the strategies that can be used are:
- Versioning using the API Version in the URI
- Versioning using a Query String
- Versioning with Custom Request Header
- Versioning with Accept Header
- Versioning with Content Type
Versioning using the API Version in the URI
One strategy of versioning the API is by including the version in the URI path. This is the most commonly used approach. For example, if we have two versions on our API, the URI for version 2 could be like this:
http://api.example.com// or using the scenario from the previous example:GET /api/v2/books
The benefit of this way of versioning is that it’s clear to the clients which is the version of the API. The negative point is that every time the version changes, it’s necessary to change the URIs.
Versioning using a Query String
Another strategy is versioning using a query string which will ask for a specific version of the API. For example:
http://api.example.com?v=2// or using the scenario from the previous example:GET /api/books?v=2
The benefit of this way of versioning is that it’s possible to use a default version of the API when the version is not specified. The negative point is that it’s too easy for clients to miss the version of the API.
Versioning with Custom Request Header
Another versioning strategy is versioning with a custom request Header. In this case, the version of the API will be in the header. For example:
GET /api/books HTTP/1.1
X-Version: 2// or:Accept-version: v1
The benefit of this way of versioning is that this approach separates versioning from the rest of the API, so who is writing the API doesn’t necessarily need to change the version of the API, it’s only necessary to update the version in the header. The negative point is that it’s a bit more complex for the developer to handle the headers in the requests (it’s necessary to know how to add and intercept the calls so that those headers can be added to their client code).
Versioning with Accept Header
Another versioning strategy is versioning with Accept Header. In this approach, instead of creating a custom header, you will use the accept header itself in order to ask for a specific version of your API. For example:
GET /api/books HTTP/1.1
The benefit of this way of versioning is that it’s not necessary to create your own custom header. The negative point is that is not so clear like using query string or version in the URI, and also demand more complexity to use.
Versioning with Content Type
Another versioning strategy is versioning with Content Type (or also the Accept Header). This is the most complex approach to implement but can be very useful. In this approach, each part of the application can be in a specific version. This is an example:
GET /api/books HTTP/1.1
The benefit of this way of versioning is that it’s possible to version the payload (a payload in API is the actual data pack that is sent with the GET method in HTTP) as well as the API call itself. The negative point is that is more complex to create and maintain.
TIP: When working with ASP.NET Core it’s possible to use a library named
Microsoft.AspNetCore.Mvc.Versioning , in order to versioning the API. And also the package
Microsoft.AspNetCore.Mvc.Versioning.ApiExplorer which can be used in order to the documentation also make use of the versioning.
Securing the API
You should always consider securing your API in cases where:
- The API will make use of private or personalized data
- The API will send sensitive data access the network
- The API will use any kind of credentials
- The API will try to protect against the overuse of your servers
Authentication vs. Authorization
In simple terms, Authentication is “who you are”. Authentication is about the information that is used to determine your identity, this may include credentials like username and password or claims, and the server can use this information to identify you and make sure that you are exactly who you say you are. Authorization is “what you can do”, based on the authentication information, based on the identity itself. Authorization is related to rules about rights, roles, permissions.
Authentication Types for APIs
The most common ways for securing an API is by:
- Basic Auth — This is also used a lot, and it is easy to implement. Basic Auth allows you to pass the information in the query string or in the headers, with the credentials to be validated on the server. This is also not so secure because can leak the username and password from the users. The credentials will be sent on every request and someone can intercept and get them.
- Token Based Auth — This is the most commonly used and has a mix of being secure and simple. There is a standard for these tokens and there is usually middleware that will support creating and validating these tokens on most platforms.
- OAuth — OAuth has a couple of standards and it is used to allow trusted third parties to identify users. This way the application that is using OAuth never gets the credential. So for example, if you want to login you can use your Gmail or Microsoft account, instead of informing username and password.
There many kinds of Token, one of the most common and more used is the JWT Token. With this approach, the token will be sent in the header of the request, like this:
This is how it works: The client will send a request to the server with his credentials (can be for example the username and password), the server will validate this authentication and will return a token, which is a series of characters. The client doesn’t need to decode this token, it will only use it to send in the requests. In a SPA application for example, the browser will store this token (in a mobile application you can store this in the cache of the app) and every time the client request something, the token will be sent in the header of the request. The server will then read this token and will validate if the user has the authorization to execute that action and will return a response:
JSON Web Token (JWT) is an open standard (RFC 7519) that defines a compact and self-contained way for securely transmitting information between parties as a JSON object. This information can be verified and trusted because it is digitally signed. JWTs can be signed using a secret (with the HMAC algorithm) or a public/private key pair using RSA or ECDSA.
A JWT token can contain:
- Information about the user
- Claims for the kind of rights it has
- Validate Signature
- Other information
To understand the structure of the token, and also to make some tests, you can use the website https://jwt.io/. On the image below, there is an example of a JWT token:
On the left side of this image, there is the JWT token itself. On the right side, there is the information related to this token.
The red part of the token is about the header. On the header, there are information about which kind of cryptography algorithm is used on this token (in this example is “HS256”) and which kind of token this token is (which in this example is a JWT token, but exist many types of token).
The purple part of the token is about the data of this token. On the right side of the picture, in the “Payload”, there are the properties “sub”, “name’ and “iat”, these information are encrypted on this token using the algorithm HS256.
The blue part of the token is about the signature, which uses a cryptographic key. This key is the key that is in your application. With this key, it’s possible to encrypt or decrypt the token.
Extra: SOAP — Simple Object Access Protocol
Before REST, another pattern that was very used was SOAP. The main difference between REST and SOAP, is that REST transfer the data using HTTP, but also use text to transfer the data (the data is sent by text, and is received by text), that's why REST is very light. SOAP also uses HTTP but is based on XML, and contains much more information than REST. This makes the message much heavier. SOAP is still being used in some situations, but in general, REST is much more used.
REST APIs is very used in many applications nowadays, it’s light, fast and really works well. Following the principles that were mentioned in this article allows you to have a good design in your APIs, but don't be so restrict on that, always take into account what does your application really need and make use of the principles that will be good for your app.
Thanks for reading!