Factoid Fizz

To run any software application, you need four key resources: CPU, memory, disks, and network. These resources are limited on a single server.

Let’s consider a scenario where you have a single server hosting your well-known application. This application receives user requests via HTTP REST API and stores data in a database after processing. We will now explore the limitations of using a single server due to these factors:

• Network Bandwidth
• Disk I/O
• CPU and Memory

Network Bandwidth#

Network bandwidth is limited—assume you have a 1 Gbps network bandwidth. ~= 125 MB/s

TPS = Bandwidth (in bytes per second) / Size of data per request (in bytes)

Single server can reduce TPS if you increase the size of data per request

• If you have 5KB of data per request, you can handle 25,000 requests per second.
• If you have 50KB of data per request, you can handle 2,500 requests per second.

Disk I/O#

Disk performance (throughput and latency) depends on:

• type of read/write operations (sequential/random)
• disk rotational speed, seek time and proper use of disk caches
• concurrent read/write operations and application-based transactional operations

These factors can limit the number of transactions per second (TPS) that your application can handle.

Disk I/O is limited—assume you have a disk that can handle 1000 IOPS (Input/Output Operations Per Second).

TPS = Disk IOPS / IOPS per request

Single server can reduce TPS if your application needs more IOPS per request

• If your application requires 1 IOPS per transaction, you can handle 1000 transactions per second.
• If your application requires 10 IOPS per transaction, you can handle 100 transactions per second.

CPU and Memory#

CPU and memory are the most critical resources for any application. CPU and memory have limits. If an application tries to use more than what’s available, it has to wait.

TPS = Total CPU cycles per second / CPU cycles per request

Assume you have a server with 1-core CPU which supports 1-million CPU cycles per second.

Single server can reduce TPS if your application needs more CPU cycles

• If your application requires 100 of CPU cycles per request, you can handle 10,000 requests per second.
• If your application requires 1000 of CPU cycles per request, you can handle 1000 requests per second.

TPS = Total memory per second / Memory per request

Assume you have a server with 32 GB of memory.

Single server can reduce TPS if your application needs more memory

• If your application requires 100 MB of memory per request, you can handle 320 requests per second.
• If your application requires 1000 MB of memory per request, you can handle 32 requests per second.

Conclusion#

A single server will always have limits on how much it can handle. To scale your application and achieve higher TPS, you need to spread the workload across several servers.