How to design a REST API? – Techno Scriber

In the modern software development landscape, designing robust and efficient APIs is paramount. Representational State Transfer (REST) has emerged as a popular architectural style for designing networked applications. REST APIs enable interoperability between different systems and provide a standardized approach for communication over the web. This article explores the principles and best practices involved in designing REST APIs.

API Basics

What is an API?

An Application Programming Interface (API) is a set of protocols, tools, and definitions that allows different software applications to communicate with each other. It defines the methods and data formats that applications can use to request and exchange information. APIs are commonly used in software development to enable integration between different systems, services, or components, allowing them to work together seamlessly.

APIs serve as intermediaries between applications, providing a standardized way for them to interact without needing to understand each other’s internal workings. They abstract away the complexity of underlying systems and expose a simplified interface that developers can leverage to perform various tasks or access resources.

API types

Web APIs: These are APIs that are accessed over the internet using standard web protocols such as HTTP. Web APIs are typically used to enable communication between web-based applications or between web applications and external services. Examples include RESTful APIs, SOAP APIs, and GraphQL APIs.
Library APIs: Library APIs, also known as programming interfaces, are sets of functions or classes provided by libraries or frameworks to enable developers to perform specific tasks or access certain functionalities. These APIs are typically used within the same programming language or environment.
Operating System APIs: Operating systems provide APIs that allow applications to interact with system resources such as files, devices, and memory. These APIs provide a way for applications to perform operations such as file I/O, network communication, and process management.

Importance of API

APIs are important because they will be used:

By the User Interface: Modern apps are built for web and mobile. Mobile apps must have an API on the server to communicate with it. Web clients built using SPA also must have an API on the server.
To extend the app’s reach: By exposing API, other apps can use your data.
To allow monetization: We can also charge for access to our data.

Why do you need a well-designed API?

For APIs, developers are the customers and they have alternatives. Our API Should be easy to use and consistent. In addition, satisfied customers mean fewer support calls. So in order to get happy developers we need a well-designed API.

Web APIs

Web API is the most popular and widely used kind of API. A web API is platform agnostic meaning a .NET component can easily work with a python component. This holds good for other platforms. In addition, Web APIs usually use standard protocols generally HTTP. Also, they are request/response based. Let us look at the different types of web APIs now:

SOAP

SOAP is an acronym for Simple Object Access Protocol. It was designed in 1998 for Microsoft. It’s an XML-based protocol. It uses a Remote Procedure Call(RPC Style). It is an extensible protocol. SOAP is very outdated and should not be used unless it has to be used.

REST

REST is an acronym for Representational State Transfer. It was designed in 2000 by Roy fielding. REST is url+JSON based message style. REST is the de facto standard for Web APIs.

GraphQL

GraphQL was developed internally in 2012 by Facebook(now Meta). It was released publically in 2015.
It enables very flexible querying, updating and subscribing to data changes. It is based on JSON in and out. GraphQL although very flexible is not very easy and requires some upfront development effort. In addition, it requires performance optimization.

gRPC

gRPC was developed by Google in 2015 and uses HTTP/2. It uses Protobuf as the transport mechanism. It supports bi-directional and streaming messaging. gRPC is quite performant but it’s not widely used yet. It requires specialized libraries at both ends.

REST Basics

REST stands for Representational State Transfer. REST API enables transfer of representation of a resource’s state.

What is REST?

At its core, REST is an architectural style that emphasizes a stateless client-server interaction model. RESTful systems are characterized by their use of standard HTTP methods such as GET, POST, PUT, DELETE, etc., to perform CRUD (Create, Read, Update, Delete) operations on resources. Additionally, REST APIs rely on a uniform interface, which typically includes the use of URIs (Uniform Resource Identifiers) to identify resources and hypermedia links to navigate between them.

Structure of REST API request

The structure of a REST API request typically follows a standardized format, which includes several key components. These components help specify the operation to be performed and provide any necessary data for the server to process the request effectively. Here’s an overview of the structure of a REST API request:
HTTP Method (Verb):The HTTP method, also known as the HTTP verb, indicates the type of operation to be performed on the resource. Commonly used HTTP methods in RESTful APIs include:

GET: Used to retrieve a representation of a resource.
POST: Used to create a new resource.
PUT: Used to update an existing resource or create a new one if it doesn’t exist.
DELETE: Used to delete a resource.
PATCH: Used to partially update a resource.

Resource URI (Uniform Resource Identifier): The resource URI specifies the location of the resource being accessed or manipulated. It uniquely identifies the resource within the API’s domain. The URI typically follows a hierarchical structure, with different segments representing different levels of the resource hierarchy.
For example:

GET /users
POST /users
PUT /users/{id}
DELETE /users/{id}

HTTP Headers: HTTP headers provide additional metadata about the request, such as the content type, authorization credentials, caching directives, and more. Some common headers used in REST API requests include:

Content-Type: Specifies the format of the request body, such as application/json or application/xml.
Authorization: Provides authentication credentials for accessing protected resources.
Accept: Indicates the desired content type for the response.
Cache-Control: Specifies caching behavior for the response.

Request Body (Payload):For operations like POST, PUT, or PATCH, the request may include a payload or body containing data to be processed by the server. The format of the request body depends on the content type specified in the Content-Type header. Common formats include JSON, XML, form-urlencoded data, and multipart form data.
For example, when creating a new user resource:

POST /users

{
"username": "john_doe",
"email": "john@example.com",
"password": "secret"
}

Query Parameters: Query parameters are used to filter, paginate, or customize the response returned by the server. They are appended to the resource URI after a question mark (?) and separated by ampersands (&). Query parameters are commonly used with GET requests to specify search criteria or pagination options.
For example:
GET /users?role=admin&limit=10&page=1

Structure of REST API Response

The structure of a REST API response typically follows a standardized format, designed to convey information about the outcome of the request and any relevant data requested by the client. Here’s an overview of the key components commonly found in a REST API response:
HTTP Status Code: The HTTP status code is a three-digit numeric code included in the response header, indicating the outcome of the request. It provides information about whether the request was successful, encountered an error, or requires further action. Some common HTTP status codes include:

200 OK: The request was successful.
201 Created: The resource was successfully created.
400 Bad Request: The request was invalid or malformed.
404 Not Found: The requested resource was not found.
500 Internal Server Error: The server encountered an unexpected error.

Response Headers: Similar to request headers, response headers provide additional metadata about the response, such as content type, caching directives, and authentication information. They convey information that supplements the response body and aids in processing the response on the client side.
Response Body (Payload):The response body contains the data returned by the server in response to the client’s request. The format of the response body depends on the content type specified in the response headers, such as JSON, XML, HTML, or plain text. The response body may include a representation of a resource, error messages, or any other relevant data requested by the client.
For example, a JSON-formatted response body might look like this:
{
“id”: 123,
“username”: “john_doe”,
“email”: “john@example.com”,
“created_at”: “2024-02-07T12:00:00Z”
}
Pagination Metadata:In cases where the response contains a large collection of resources, pagination metadata may be included to indicate the total number of items, the current page, and links to navigate between pages. This helps clients navigate through paginated results efficiently.
Hypermedia Links (HATEOAS):Following the principles of HATEOAS (Hypermedia as the Engine of Application State), RESTful APIs may include hypermedia links in the response to enable clients to navigate the API dynamically. These links provide URLs to related resources or actions that clients can follow to interact with the API further.

HTTP Verbs

RESTful APIs utilize a set of HTTP methods, often referred to as HTTP verbs, to define the actions that clients can perform on resources. These HTTP methods correspond to CRUD (Create, Read, Update, Delete) operations and provide a standardized way to interact with resources. Here are the commonly used HTTP verbs in RESTful APIs:

The GET Verb

The GET method is used to retrieve a representation of a resource or a collection of resources from the server. It is a safe and idempotent operation, meaning that it should not modify the state of the server and can be repeated multiple times without side effects.

GET /users
GET /users/{id}

The POST Verb

The POST method is used to create a new resource on the server. It typically involves sending data in the request body, which the server uses to create the resource. POST requests may result in the creation of a new resource, the modification of existing resources, or the execution of server-side actions.
Example:

POST /users
POST /orders

The PUT Verb

The PUT method is used to update an existing resource or create a new one if it doesn’t exist. It replaces the entire representation of the resource with the new data provided in the request body. PUT requests are idempotent, meaning that repeating the same request multiple times has the same effect as a single request.

PUT /users/{id}

The DELETE Verb

The DELETE method is used to delete a resource from the server. It instructs the server to remove the specified resource, if it exists. Like GET requests, DELETE requests are idempotent, meaning that repeating the same request multiple times has the same effect as a single request.
Example:

DELETE /users/{id}

The PATCH Verb

The PATCH method is used to partially update an existing resource. It allows clients to send only the data that needs to be modified, rather than the entire representation of the resource. PATCH requests are often used when updating specific fields or properties of a resource.
Example:

PATCH /users/{id}

URL Structure

The URL structure of a REST API plays a crucial role in defining the endpoints through which clients can interact with the resources exposed by the API. A well-designed URL structure follows certain conventions and provides a clear and intuitive way for clients to access and manipulate resources. Here are some key considerations and best practices for structuring REST API URLs:
Use Nouns to Represent Resources: RESTful APIs should use nouns to represent resources rather than actions. Resource names should be descriptive and reflect the entities or objects being manipulated by the API. This promotes clarity and consistency in the API design.
Example:

/users
/posts
/comments

Use Plural Nouns for Collections: Resource URIs representing collections of resources should use plural nouns to indicate that they represent multiple instances of the resource.
Example:

/users
/posts
/comments

Use Singular Nouns for Individual Resources: Resource URIs representing individual resources should use singular nouns followed by a unique identifier (such as an ID or slug) to specify a specific instance of the resource.
Example:

/users/{id}
/posts/{id}
/comments/{id}

Hierarchical Structure for Nested Resources: If resources have a hierarchical relationship, the URL structure can reflect this hierarchy by using nested resource paths. Each level of nesting represents a relationship between resources.
Example:

/users/{userId}/posts
/posts/{postId}/comments

Avoid Verb Usage in URIs: URIs should focus on identifying resources rather than specifying actions to be performed on them. Use HTTP methods (GET, POST, PUT, DELETE, etc.) to indicate the action to be performed on a resource.
Incorrect:

/getUsers
/createPost

Correct:

GET /users
POST /posts

Use Hyphens or Underscores for Readability: Use hyphens or underscores to separate words within resource names for improved readability. This can make URIs more human-readable and easier to understand.
Example:

/user-profiles
/post-comments

Versioning:Consider incorporating versioning into the URL structure to support backward compatibility as the API evolves over time. Versioning helps ensure that clients can continue to use older versions of the API while transitioning to newer versions.
Example:

/api/v1/users
/api/v2/posts

Response codes

RESTful APIs utilize HTTP status codes to communicate the outcome of a client’s request to the server. These status codes provide information about whether the request was successful, encountered an error, or requires further action. Understanding and properly handling HTTP status codes is essential for clients to interpret and respond to API responses effectively. Here are some common REST API response codes along with their meanings:

2xx Success

200 OK: Indicates that the request was successful, and the server has returned the requested resource(s) in the response body.
201 Created: Indicates that the request was successful, and a new resource has been created as a result of the request. The URI of the newly created resource is typically included in the response headers.
204 No Content: Indicates that the request was successful, but there is no content to return in the response body. This is often used for successful DELETE requests.

3xx Redirection:

301 Moved Permanently: Indicates that the requested resource has been permanently moved to a new location. The client should use the new URI provided in the response to access the resource in the future.
302 Found: Indicates that the requested resource has been temporarily moved to a different URI. The client should use the new URI provided in the response for the current request, but future requests may still use the original URI.

4xx Client Error:

400 Bad Request: Indicates that the request sent by the client is malformed, invalid, or otherwise incorrect, and the server cannot process it.
401 Unauthorized: Indicates that the request requires authentication, and the client must provide valid credentials to access the requested resource.
403 Forbidden: Indicates that the server understood the request but refuses to authorize it. The client does not have permission to access the requested resource.
404 Not Found: Indicates that the requested resource could not be found on the server.

5xx Server Error:

500 Internal Server Error: Indicates that an unexpected error occurred on the server while processing the request. This could be due to a bug in the server-side code or an external dependency failure.
502 Bad Gateway: Indicates that the server received an invalid response from an upstream server while acting as a gateway or proxy.
503 Service Unavailable: Indicates that the server is temporarily unable to handle the request due to maintenance, overloading, or other transient issues.

REST API Documentation

REST API documentation plays a crucial role in guiding developers on how to use an API effectively. It provides comprehensive information about the API’s endpoints, request and response formats, authentication mechanisms, error handling, and other relevant details. Well-documented APIs facilitate integration, improve developer experience, and promote adoption by making it easier for developers to understand and interact with the API.

Introduction to OpenAPI

OpenAPI, formerly known as Swagger, is a widely adopted standard for describing and documenting RESTful APIs. It provides a machine-readable format for defining the structure, endpoints, parameters, request and response formats, and other details of an API in a standardized way. OpenAPI specifications are written in JSON or YAML format and serve as a contract between API providers and consumers, defining how the API should be used and what behaviour can be expected.

OpenAPI features

API Description: OpenAPI allows developers to describe the structure and functionality of their APIs using a clear and standardized format. This includes defining endpoints, operations, request parameters, response formats, error codes, and other relevant details.
Machine-Readable Format: OpenAPI specifications are written in JSON or YAML, making them easy to parse and consume by both humans and machines. This machine-readable format enables automated tooling for generating documentation, client libraries, and server stubs from the API specification.
Interactive Documentation: OpenAPI specifications can be used to generate interactive API documentation, providing developers with a user-friendly interface for exploring and testing API endpoints directly within the documentation. This helps improve the developer experience and accelerates API adoption.
Client Code Generation: OpenAPI specifications can be leveraged to generate client libraries in various programming languages, allowing developers to easily integrate with the API in their preferred language and environment. Client code generation automates the process of mapping API endpoints to method calls and serializing/deserializing request and response data.
Server Code Generation: In addition to client code generation, OpenAPI specifications can also be used to generate server stubs, providing a scaffold for implementing the API endpoints and handling incoming requests. Server code generation helps reduce development effort and ensures consistency between the API specification and its implementation.
API Testing: OpenAPI specifications can be used to generate API test cases, enabling automated testing of API endpoints against the expected behaviour defined in the specification. This helps ensure that the API functions correctly and remains consistent with the documented behaviour over time.

Authentication and Authorization

Authentication is the process of verifying the identity of a user or client who is attempting to access an API. It involves presenting credentials (such as usernames, passwords, API keys, or tokens) to the server, which then validates the credentials to authenticate the user’s identity.

The OAuth Protocol

OAuth (Open Authorization) is an open standard for token-based authentication and authorization that is widely used in modern RESTful APIs to delegate access to protected resources on behalf of a user. It enables third-party applications to obtain limited access to a user’s resources without requiring the user to share their credentials directly with the third party. The OAuth protocol typically involves four main participants:
Resource Owner: The resource owner is the user who owns the protected resources (such as data or services) and is capable of granting access to those resources to other parties.
Client: The client is the application or service that is requesting access to the user’s resources. It could be a web application, mobile app, or other software components.
Authorization Server: The authorization server is responsible for authenticating the resource owner and issuing access tokens to authorized clients. It verifies the identity of the resource owner and determines whether the client is allowed to access the requested resources.
Resource Server: The resource server hosts the protected resources that the client wants to access. It validates the access tokens issued by the authorization server and grants or denies access to the requested resources based on the token’s validity.

The JWT

JWT (JSON Web Token) is a compact, URL-safe means of representing claims to be transferred between two parties. These claims are typically used to provide information about an authenticated user or to assert some information about a user’s session. JWTs are commonly used in RESTful APIs for authentication and authorization purposes due to their flexibility, scalability, and statelessness.

JWT in Rest API

Here’s how JWTs are used in REST APIs:
Token Generation (Authentication):

When a user authenticates with the API (e.g., by providing valid credentials), the server generates a JWT containing relevant user information (claims) such as user ID, username, and any other necessary data.

The JWT is signed using a secret key known only to the server, ensuring that it cannot be tampered with by clients.

The server sends the JWT back to the client as part of the authentication response (typically in the response body or headers).

Token Verification (Authorization):

Subsequent requests from the client to access protected resources include the JWT in the request headers (commonly in theAuthorizationheader as a Bearer token).

The server verifies the JWT’s signature using the secret key to ensure its integrity and authenticity. If the signature is valid, the server extracts and decodes the claims from the JWT.

The server performs authorization checks based on the claims extracted from the JWT. This may include checking the user’s permissions, roles, or any other relevant information contained in the JWT.

If the JWT is valid and the user is authorized to access the requested resource, the server fulfills the request. Otherwise, it returns an appropriate error response (e.g., 401 Unauthorized or 403 Forbidden).

Statelessness and Scalability:

JWTs are stateless, meaning that all necessary information to authenticate and authorize a user is contained within the token itself. This eliminates the need for the server to store session state, making JWT-based authentication more scalable and suitable for distributed systems.

Since JWTs are self-contained and contain all necessary information, they can be easily passed between microservices or API gateways without the need for shared session state.

Token Expiry and Refresh:

JWTs can include an expiration (exp) claim, specifying a timestamp after which the token is no longer valid. This helps mitigate the risk of token replay attacks and unauthorized access.

To handle token expiry, clients can refresh JWTs by obtaining a new token using a refresh token (if supported by the authentication mechanism) or by re-authenticating with the server.

Custom Claims and Authorization Context:

JWTs can include custom claims that provide additional context or information about the user’s session, such as user roles, permissions, or other attributes.

API servers can leverage these custom claims to make fine-grained authorization decisions based on the user’s attributes and roles.

Performance

Optimizing the performance of a RESTful API is crucial for ensuring fast response times, scalability, and efficient resource utilization, leading to improved user experience and reduced server load. Here are some key strategies for optimizing the performance of a REST API:
Use Efficient Data Formats:

Choose lightweight and efficient data formats such as JSON (JavaScript Object Notation) or Protocol Buffers for representing data exchanged between clients and servers.

Minimize unnecessary data in responses by only including relevant fields and avoiding nested or redundant structures.

Consider compressing response payloads using techniques like gzip to reduce bandwidth usage and improve network efficiency.

Implement Caching:

Utilize caching mechanisms to store frequently accessed or static data at various layers of the application stack (e.g., in-memory cache, CDN caching, browser caching).

Leverage HTTP caching headers (e.g., Cache-Control, ETag, Last-Modified) to control caching behavior and reduce server load by allowing clients to cache responses locally.

Optimize Database Queries:

Use efficient database queries and indexes to minimize database load and reduce query execution times.

Employ techniques like query optimization, indexing, denormalization, and query caching to improve database performance and scalability.

Consider implementing data caching mechanisms (e.g., Redis, Memcached) to cache frequently accessed database records and reduce latency.

Implement Asynchronous Processing:

Offload long-running or resource-intensive tasks to background workers or asynchronous processing frameworks.

Use message queues (e.g., RabbitMQ, Apache Kafka) to decouple components and distribute workload across multiple workers, improving scalability and responsiveness.

Optimize Network Requests:

Minimize the number of HTTP requests required to fulfill a client’s request by consolidating multiple API calls into a single request whenever possible.

Utilize HTTP/2 to take advantage of features like multiplexing, header compression, and server push, reducing latency and improving network performance.

Use Content Delivery Networks (CDNs):

Distribute static assets (e.g., images, CSS, JavaScript) through a CDN to cache content closer to users and reduce latency for geographically distributed clients.

Leverage CDN caching capabilities to offload traffic from the origin server and improve overall scalability and availability.

Implement Rate Limiting and Throttling:

Enforce rate limiting and throttling mechanisms to prevent abuse, mitigate DDoS attacks, and ensure fair usage of API resources.

Use techniques like token bucket algorithms or leaky bucket algorithms to control the rate of incoming requests and prevent API abuse.

Monitor and Analyze Performance:

Implement monitoring and logging solutions to track API performance metrics (e.g., response time, throughput, error rates) and identify performance bottlenecks.

Use performance profiling tools and APM (Application Performance Monitoring) solutions to analyze API performance and optimize critical paths.

Horizontal Scaling:

Scale the API horizontally by adding more server instances or deploying microservices to distribute load and improve scalability.

Utilize container orchestration platforms (e.g., Kubernetes, Docker Swarm) to automate deployment, scaling, and management of API instances.

Optimize Security Measures:

Ensure that security measures, such as authentication and authorization checks, are implemented efficiently without introducing unnecessary overhead.

Employ techniques like token-based authentication, JWT validation, and access control lists (ACLs) to minimize the impact on performance while maintaining security.

Advanced Topics

HATEOAS

HATEOAS (Hypermedia as the Engine of Application State) is a constraint in the REST architectural style that emphasizes the use of hypermedia links to navigate the API’s resources dynamically. In essence, HATEOAS enables clients to discover and interact with resources by following links embedded within the representation of the resources themselves. This approach decouples clients from specific URLs and provides a more flexible and self-descriptive API architecture. Here’s how HATEOAS works in REST APIs:

Hypermedia Links:

Instead of hardcoding URLs into client applications, REST APIs following HATEOAS principles include hypermedia links within the representations of resources.

Hypermedia links (also known as “hyperlinks” or “HREFs”) are URLs embedded in the response body of API endpoints, allowing clients to navigate to related resources or perform actions by following these links.

Dynamic Resource Navigation:

Clients can dynamically navigate the API’s resources by following hypermedia links returned in API responses.

Each resource representation includes links to related resources or actions that clients can take next, based on the current state of the application.

Clients do not need to have prior knowledge of API endpoints or URL structures; they can discover available actions and transitions by inspecting the links provided in responses.

Stateless Interaction:

HATEOAS promotes the statelessness of RESTful interactions by encapsulating all necessary information for navigating the API within the representations of resources.

Clients maintain no client-specific state related to the API server, allowing them to interact with the API in a stateless manner.

This simplifies client-server interactions and enables better scalability and fault tolerance in distributed systems.
Enhanced Discoverability and Flexibility:

HATEOAS enhances the discoverability of APIs by providing a self-descriptive interface that exposes available actions and transitions to clients.

Clients can evolve independently of server changes, as they rely on hypermedia links rather than hardcoded URLs.

APIs become more flexible and adaptable to changes, as modifications to the server’s resource structure or URL paths do not require corresponding updates to client applications.
Standard Formats:

HATEOAS implementations often use standard hypermedia formats such as HAL (Hypertext Application Language), JSON-LD (JSON Linked Data), or Siren (Simple Hypermedia Type) to define the structure of hypermedia representations and links.

These formats provide conventions for representing hypermedia links, embedded resources, and other metadata in a standardized way.

API Gateway and Discovery

In RESTful API architectures, an API gateway and service discovery play important roles in managing and orchestrating the interactions between clients and microservices. Here’s an overview of each concept:

API Gateway: An API gateway is a centralized entry point for clients to access various microservices within a system. It acts as a reverse proxy that handles client requests, routing them to the appropriate microservice(s), aggregating responses, and providing additional features such as authentication, rate limiting, caching, and logging. Key features and benefits of an API gateway include:

Routing: The API gateway routes client requests to the appropriate microservices based on the request URL, HTTP method, or other criteria defined in routing rules.
Authentication and Authorization: The API gateway handles authentication and authorization of client requests, enforcing access control policies and validating user credentials before forwarding requests to backend services.
Load Balancing: An API gateway can distribute incoming requests across multiple instances of microservices to achieve load balancing and improve scalability.
Security: The API gateway provides a centralized point for enforcing security measures such as HTTPS encryption, CSRF protection, and API key management.
Monitoring and Analytics:API gateways often include monitoring and analytics features to track API usage, performance metrics, error rates, and other relevant data.

Service Discovery: Service discovery is the process of dynamically locating and identifying instances of microservices within a distributed system. In a microservices architecture, where services may be deployed and scaled independently, service discovery enables clients and services to locate each other without hardcoded dependencies or static configurations. Key aspects of service discovery include:

Registration: Microservices register themselves with a service registry (such as Consul, Eureka, ZooKeeper) when they start up, providing metadata such as service name, host, port, and health status.
Discovery: Clients and other microservices use the service registry to discover available instances of services by querying the registry based on service names or other criteria.
Dynamic Updates: Service registries support dynamic updates, allowing microservices to register, deregister, and update their metadata in real-time as they scale up, scale down, or undergo maintenance.
Load Balancing: Service discovery mechanisms often include built-in load balancing capabilities to distribute client requests among multiple instances of a service, improving fault tolerance and scalability.
Health Checking:Service registries perform health checks on registered services to monitor their availability and responsiveness, removing unhealthy instances from the registry to prevent client requests from being routed to them.