Journey to Optimize Network Usage (Part 1)

BACKND

18 Feb 2025 — 8 min read

Written by Devops Traveler Tales, 18 February 2025

All resources are limited. We build our services using these finite resources, and we pay for every bit of them.

Today, I’d like to talk about these limited resources and their associated costs—specifically, network usage.

Among the many resources, network usage is notably expensive and, during our system optimization efforts, emerged as the resource with the most potential for improvement.

This article is divided into two parts. In Part 1, the factors that drive network costs and why these costs can feel high will be discussed. In Part 2, I’ll share anecdotes from our service—detailing where significant network costs occurred and the efforts we undertook to reduce them.

When Do Network Costs Occur?

Let’s first cover when network costs are incurred.

There are two primary components of network costs:

On-Demand Costs : These are usage-based fees—for example, the per-GB charges specified in the pricing policies for AWS load balancers.
Resource Purchase Costs : These refer to the cost of purchasing the resources themselves, such as the network capacity provided with an EC2 instance. More details will be explained in the below on this shortly.

On-demand network costs are measured in various ways. When traffic leaves AWS to the external internet—via services like CloudFront or ELB—costs are applied accordingly. Additionally, if services like EC2, Lambda, or Kinesis send data out to the internet during processing, fees accrue.

Even communication within AWS incurs network costs when traffic is transferred to another AZ(Availability Zone) or Region. Moreover, when you manage your network via a VPC, using services like VPC Peering or Transit Gateway can also add to the cost. [1]

The second component of network costs is the cost associated with purchasing network resources. In infrastructure, resources such as CPU and memory are available but inherently limited. At first glance, it might be unclear when these costs manifest, but in reality, they occur very frequently—especially when using EC2 instances.

For example, a single c6i.xlarge instance provides 4 CPU cores, 8 GB of memory, and up to 12.5 gigabits of network traffic. System engineers often liken these resources to a “bag” with compartments: purchasing a c6i.xlarge bag means you obtain one with 4 CPU compartments, 8 memory compartments, and up to (?) 12.5 compartments for network traffic!

We then fill these compartments with workloads—commonly referred to as servers. For instance, if a server uses 1 CPU core and 2 GB of memory, you could fit up to 4 servers in that bag; any additional servers would require another bag.

But here’s the nuance: when you purchase a c6i.xlarge bag, the statement “up to (?) 12.5 compartments for network traffic” requires careful interpretation. Without understanding what “up to” means, one might misinterpret the available network resources.

Looking at the resource specifications for AWS EC2 instances, you’ll notice that a maximum network bandwidth is defined—indicating that this bandwidth operates on an I/O credit basis.

The credit mechanism is a common model used by AWS. There is a baseline, and any usage below that, and any time you use less than that, you get credit for the unused portion —up to a maximum xxx GB limit.

When a burst of requests exceeds the baseline (= a burst usage), the accumulated credits are consumed, reducing the credit balance. If a burst occurs when no credits are available, the requests cannot be processed, leading to network throttling.

To check the baseline bandwidth of EC2 instances—which are documented with a maximum bandwidth —you can use the AWS CLI. For example, the following command retrieves details for c6i instances [2].

aws ec2 describe-instance-types \
    --filters "Name=instance-type,Values=c6i.*" \
    --query "InstanceTypes[].[InstanceType, NetworkInfo.NetworkPerformance, NetworkInfo.NetworkCards[0].BaselineBandwidthInGbps] | sort_by(@,&[2])" \
    --output table

The output from the AWS CLI looks something like this:

-------------------------------------------------
|             DescribeInstanceTypes             |
+---------------+----------------------+--------+
|  c6i.large    |  Up to 12.5 Gigabit  |  0.781 |
|  c6i.xlarge   |  Up to 12.5 Gigabit  |  1.562 |
|  c6i.2xlarge  |  Up to 12.5 Gigabit  |  3.125 |
|  c6i.4xlarge  |  Up to 12.5 Gigabit  |  6.25  |
|  c6i.8xlarge  |  12.5 Gigabit        |  12.5  |
|  c6i.12xlarge |  18.75 Gigabit       |  18.75 |
|  c6i.16xlarge |  25 Gigabit          |  25.0  |
|  c6i.24xlarge |  37.5 Gigabit        |  37.5  |
|  c6i.32xlarge |  50 Gigabit          |  50.0  |
|  c6i.metal    |  50 Gigabit          |  50.0  |
+---------------+----------------------+--------+

The third column indicates the baseline bandwidth. For the frequently used c6i.xlarge instance, the baseline network bandwidth is 1.562 Gbps. Since Gbps is measured in bits, converting to bytes (as engineers often do) gives 1.562/8 = 0.19525 GBps—that is, approximately 199.936 MBps of network bandwidth.

Furthermore, assume that this bandwidth represents both inbound and outbound. If a workload uses these equally, each direction gets roughly 99.968 MBps. And if you assume that 10 servers are running on the same instance, each server would share about 10 MB/s of network bandwidth.

Just as you must purchase additional instances when CPU or memory is insufficient, if your network usage exceeds the available bandwidth, you'll need to purchase additional instances. In this way, a portion of the cost of each instance is inherently tied to network expenses.

Great! It seems we've covered all aspects regarding the components of network costs.

Are Network Costs Expensive?

The receipt that might make you want to tear it up

It can be seen both ways. First, while AWS provides cloud infrastructure, it isn’t an ISP that directly sets up local internet networks. Instead, AWS relies on networks set up by local ISPs—and as a result, you pay for using that network. Consequently, the rates established by the local ISP have a direct impact on network costs.

Based on this logic, network costs vary by AWS region. For example, the Transit Gateway costs $0.07 per connection in the Seoul region, whereas in the Oregon region the cost is only $0.05 per connection. Similarly, EC2's outbound internet cost is $0.126 per GB for the first 10TB in the Seoul region, compared to just $0.09 per GB in the Oregon region. In this way, the unit cost for network usage differs by region, leading to real cost variations depending on where your service is hosted.

On the other hand, while certain aspects of network costs are genuinely high, there's also an element of illusion. It’s much like when you total up your credit card spending at year-end—the final sum may seem exorbitant, but it’s merely the actual amount you spent.

AWS, as a cloud provider, has meticulously established a billing structure that charges us precisely for every bit of network usage. Essentially, we pay exactly for what we use—but sometimes it might feel like we’re spending far more than we ever anticipated.
And indeed, in many cases, that’s true. While spending can vary depending on the expertise of the person in charge within each organization, if you’re not paying close attention, it’s easy to lose track of how much you’re spending on network usage. Consequently, the bill can seem exorbitant at first glance.

However, if you truly understand your network spending and continuously monitor your costs, receiving the bill might not be so surprising after all

—assuming that keeping track of it is straightforward.

Original link to the AWS network cost structure summary by Duckbill Group Original Source

This diagram, as summarized by the Duckbill Group, illustrates AWS’s network cost structure. Explaining every detail of the diagram falls outside the scope of this article. However, it clearly shows that AWS’s network billing is both complex and meticulous—and beginning to understand these details is the first step toward optimizing network costs.

In other words, one of the reasons network costs can become so high is not only that network usage is inherently expensive, but also that the steep learning curve imposed by such a complex billing structure contributes significantly to the overall expense.

Earlier, I explained how a portion of an individual instance’s cost is indirectly attributed to network expenses. Although this cost isn’t directly itemized as “network cost,” if increased network usage forces you to purchase additional instances, that extra expenditure can be attributed to network usage. To resolve this issue, you need to reduce network usage.

Previously, I used the analogy of a bag and its compartments to illustrate how workloads—i.e., servers—occupy these compartments. However, the resource requirements of these servers don’t always match the compartment sizes perfectly; sometimes, certain compartments end up being used more heavily than others.o

For example, when you purchase a c6i.xlarge instance, you obtain a “bag” that contains 4 CPU cores (4,000 millicores), 8 GB of memory, and a network bandwidth of 1.562 Gbps (approximately 199.936 MBps). In a scenario where a server uses only 100 millicores of CPU and 4 GB of memory, you can fit only 2 such servers in the bag.

As a result, the CPU compartment would have 3,800 millicores of unused capacity—leading to wasted resources. This inefficiency in resource utilization is commonly referred to as resource fragmentation. Resource fragmentation typically occurs when one compartment within the bag is disproportionately large or small, or when a server’s resource requirements are unbalanced.

Then, under what circumstances does network bandwidth become saturated? Typically, servers transmit text. In the realm of text, 1 megabyte is an enormous amount—for instance, it would take a 500-page book with 2,000 characters per page to equal 1 MB, meaning a server could theoretically transmit the equivalent of 20 such books per second.

However, the situation changes drastically with images, complex binaries, or videos. Installation files, for example, can easily reach 100 MB, and images or videos consume far more space than text. Consequently, when a server directly transmits such data, it uses a significant portion of its available network bandwidth.

Moreover, a server that transmits images typically retrieves the data from external storage via the network before responding, which can momentarily double the bandwidth usage.

Another scenario arises when there's a high volume of requests. Even when transmitting text, if a substantial amount of data is sent to many clients at very high speeds, the bandwidth can quickly become saturated—especially in systems where requests surge during certain periods.

The solution to these issues is well known among system engineers; regularly monitoring the workload can help prevent such problems. These challenges are often encountered by SaaS startups that initially fail to anticipate their customers’ workloads
—just as was the case with us.

Now, we're in a position to share anecdotes about the situations in which we incurred high network costs and the optimizations we implemented. However, in this post, before diving into those stories, we first explored the different types of network costs and the circumstances under which they can become expensive.