Horizontal vs. Vertical Scaling: Which Is Best for APIs?

Horizontal scaling, also known as “scaling out,” is the process of deploying additional virtual machines (VMs) so there will be more API capacity to handle an increased load. (Shrinking capacity is known as “scaling in.”) As more capacity is needed, system admins can add more VMs to the cluster. Specialized resource management software is required, however, to manage the load of API calls and route them to the right VM instance in the cluster and maintain balance.

Vertical scaling is the process of adding resources to a single node. It stands in contrast to horizontal scaling, which adds nodes. Vertical scaling, also called scaling up or scaling down, means adding resources such as central processing unit (CPU) capacity, memory, or storage to a server. In the case of APIs, vertical scaling usually involves adding computing capacity to the VM that hosts the API.

For example, if an API is hosted on a VM that’s been allocated one CPU core and 512 megabytes of random access memory (RAM), then scaling up that API could mean doubling the core count and RAM. The API would then have two dedicated CPU processor cores and 1,024 megabytes of RAM. With this new configuration, the API should be able to handle roughly double the load — though constraints on network bandwidth, storage speed, and other factors may reduce the impact of vertical scaling. There is also a resource management challenge with vertical scaling, but specialized software can typically handle this issue.

APIs are typically designed as stateless, meaning they do not store request data or retain information between sessions. This stateless nature is crucial when considering scaling methods. Stateless APIs do not require data replication across instances, making horizontal scaling more efficient and easier to implement. System admins can add or remove VMs as needed without affecting API operations, as the API client does not rely on specific server instances to function.

There is nothing wrong with being stateful. Indeed, it may be essential to the desired functioning of the app. However, it is significantly more complicated to execute horizontal scaling for a stateful app. Doing so would require copying stored data from the original version of the app to new instances.

A stateless app or API, in contrast, is one that does not store request data. It does not hold onto session data in memory. Each time a session starts, it’s as if the app is meeting the client for the first time. After the session is over, it’s “goodbye,” with no memory of the session.

Horizontal scaling is possible for a stateless app because it doesn’t matter which VM is responding to API calls. The API client can call on an infinite number of VMs hosting the API, and it will never matter. System admins can add or remove as many VMs as they want without affecting the operation of the API.

Given that APIs are stateless, horizontal scaling emerges as the right way to scale them. Adding more VMs to increase capacity works well with stateless APIs. Admins can create VM clusters that scale out as API demand grows.

Furthermore, while it is possible to scale APIs vertically, horizontal scaling is preferable because the resource allocation issues in vertical scaling make it comparatively harder to do. In contrast, horizontal autoscaling works easily with APIs. As systems management tools detect a spike in API traffic, they can automatically add VMs to host more instances of the API in a cluster. This is quite a bit more difficult in a vertical scaling situation, as autoscaling does not work well in vertical scaling.

Demand for APIs will inevitably change over time. It will be necessary to scale API capacity up or down. Horizontal and vertical scaling are both available options for APIs. However, the stateless nature of APIs, coupled with the relative ease of horizontal autoscaling, favors horizontal scaling as the right approach to scaling up APIs.

In a distributed system, where APIs are part of a microservices architecture, horizontal scaling is particularly advantageous. Each microservice can be scaled independently based on its specific load and requirements. This modular approach allows for more efficient use of resources and better fault tolerance, as failures in one microservice do not impact others.

Moreover, in a microservices architecture, horizontal scaling aligns with the principles of distributed systems, where different services are deployed across multiple nodes. This ensures that the system can handle varying loads and maintain optimal performance even during peak times.

Downtime is a critical concern when scaling APIs, as any interruption in service can lead to lost revenue and customer dissatisfaction. Horizontal scaling minimizes downtime by allowing system admins to add or remove servers without impacting the API’s availability. In contrast, vertical scaling may require rebooting the server or taking it offline temporarily, which can disrupt services.

API providers can leverage horizontal scaling to ensure continuous operation, particularly during planned maintenance or unexpected traffic spikes. By distributing the load across multiple servers, horizontal scaling ensures that the API remains responsive and available, even if individual servers need to be taken offline for upgrades or repairs.