loke.dev
Header image for Stop Increasing Your `ulimit` (Audit Your Ephemeral Port Range Instead)

Stop Increasing Your `ulimit` (Audit Your Ephemeral Port Range Instead)

Unmask the hidden network bottleneck that causes high-concurrency Node.js apps to crash with 'EADDRNOTAVAIL' long before they ever hit their actual file descriptor limits.

· 7 min read

I’ve spent more time than I’d like to admit watching a perfectly healthy Node.js process suddenly choke on its own networking while CPU and memory usage remain suspiciously low.

You know the drill: the traffic spikes, the latency climbs, and suddenly your logs are drowning in Error: connect EADDRNOTAVAIL. The first instinct, naturally, is to reach for the ulimit. We’ve been conditioned to think that "too many open files" is the only ceiling in a Linux environment. You bump the limit from 1024 to 65535, restart the service, and for a few minutes, everything looks fine. Then, like clockwork, the errors return.

The truth is that ulimit is often a red herring. While it controls how many file descriptors a process can handle, it does nothing to address the physical limitations of the TCP stack. You aren't running out of "files"; you're running out of addresses. You've hit the ephemeral port exhaustion wall.

The Anatomy of the 4-Tuple

To understand why your app is dying, we have to look at how a connection is uniquely identified in the eyes of the Linux kernel. Every TCP connection is defined by a 4-tuple:

1. Source IP (Your server)
2. Source Port (A temporary, "ephemeral" port)
3. Destination IP (The API, Database, or Microservice you're calling)
4. Destination Port (Usually 80, 443, or 5432)

When your Node.js app makes an outbound request to a database, it doesn't just "connect." The kernel has to assign a source port to that specific connection. Because your Source IP is fixed, and your Destination IP/Port (the database) is also fixed, the only variable the kernel can change to create a unique 4-tuple is the Source Port.

Here is the kicker: ports are a 16-bit unsigned integer. That means there is a hard maximum of 65,535 ports available on any given IP address. But the kernel doesn't give you all of them. It reserves a specific range for outbound connections.

How to Check Your Current Ceiling

Before you change a single line of code, you need to see what the kernel is actually allowing. On most Linux distributions, you can find your ephemeral port range by checking a specific sysctl parameter.

Open your terminal and run:

cat /proc/sys/net/ipv4/ip_local_port_range

On a default Ubuntu or Debian install, you’ll likely see something like:
32768 60999

That is your bottleneck. In this scenario, you only have 28,231 ports available for outbound connections. If your Node.js app tries to open the 28,232nd concurrent connection to the same destination, the kernel will shrug its shoulders and throw an EADDRNOTAVAIL error. It literally cannot find an "address" (a port) to make the connection "available."

The Ghost of Connections Past: TIME_WAIT

"But I'm not running 28,000 concurrent connections!" I hear this a lot. You might look at your dashboard and see only 500 active users. Why are you hitting the limit?

The answer lies in the TCP state machine. When a TCP connection is closed gracefully, it doesn't just disappear. It enters a state called TIME_WAIT. By default, the Linux kernel keeps the port "reserved" in this state for 60 seconds (twice the Maximum Segment Lifetime, or 2MSL).

This is a safety mechanism. It ensures that any "stray" packets still wandering around the internet from the old connection don't accidentally get delivered to a new connection that happens to be using the same port.

If you are doing 500 requests per second, and each one uses a new port that stays in TIME_WAIT for 60 seconds, you are effectively "consuming" 30,000 ports.

500 requests/sec * 60 seconds = 30,000 ports in use.

You just hit the wall. Your ulimit could be a million, and it wouldn't save you.

Reproducing the Crash

Let’s look at a quick Node.js snippet that demonstrates this. This script attempts to hammer an endpoint without any connection pooling.

const http = require('http');

// This mimics a high-concurrency environment without Keep-Alive
const options = {
  hostname: '127.0.0.1',
  port: 8080,
  path: '/',
  method: 'GET',
  agent: false // This forces a new connection for every single request
};

function makeRequest() {
  const req = http.request(options, (res) => {
    res.on('data', () => {});
    res.on('end', () => {
      // Port is now entering TIME_WAIT
    });
  });

  req.on('error', (e) => {
    console.error(`Broke the internet: ${e.message}`);
  });

  req.end();
}

// Blast the server
setInterval(makeRequest, 1);

If you run this against a local server, you’ll see it work for a few seconds, and then the EADDRNOTAVAIL errors will start rolling in. The kernel is literally out of breath.

Fixing the Bottleneck: The Hierarchy of Solutions

There are three ways to fix this. You should usually do them in this order.

1. Enable HTTP Keep-Alive (The Most Important Step)

The most common reason developers hit port exhaustion in Node.js is that they aren't reusing connections. By default, the standard http and https modules in older versions of Node.js (and many third-party libraries) did not enable Keep-Alive.

When Keep-Alive is on, the connection stays open after the request finishes. The next request to the same destination can "reuse" that existing 4-tuple, meaning no new ephemeral port is needed.

In modern Node.js (v19+), keepAlive is enabled by default. But if you are on an older LTS or using specific clients, you need to configure your agent:

const http = require('http');

const keepAliveAgent = new http.Agent({
  keepAlive: true,
  maxSockets: 100, // Limit how many concurrent sockets per origin
  maxFreeSockets: 10,
  timeout: 60000 // Keep idle sockets open for 1 minute
});

const options = {
  hostname: 'api.example.com',
  port: 443,
  agent: keepAliveAgent // Use the pool!
};

2. Expanding the Kernel's Horizon

If you've implemented pooling and you’re *still* hitting limits because your scale is just that massive, it’s time to talk to the kernel. You can widen the ephemeral port range to the theoretical maximum.

Edit /etc/sysctl.conf and add:

# Allow the kernel to use ports from 1024 to 65535
net.ipv4.ip_local_port_range = 1024 65535

Then apply it: sysctl -p.

By doing this, you've increased your "budget" from ~28k ports to ~64k ports. It’s a significant breathing room, but it’s still finite.

3. Tuning TCP Reuse

There is a kernel setting called tcp_tw_reuse. When enabled, the kernel allows the system to "recycle" a port that is currently in the TIME_WAIT state if it is safe to do so from a protocol perspective.

A warning: Never use tcp_tw_recycle (note the different name). It was removed in Linux 4.12 because it breaks when users are behind NAT (like almost everyone on a mobile network or in an office).

To enable safe reuse, add this to /etc/sysctl.conf:

net.ipv4.tcp_tw_reuse = 1

This is often the "silver bullet" for high-traffic proxies or API gateways that are opening and closing thousands of connections a second to a single backend.

The "Multi-IP" Strategy (For the 1%)

If you are running a massive scraper, a high-frequency trading bot, or a global-scale proxy, even 64,000 ports might not be enough. Remember the 4-tuple? If you add a second Source IP to your server, you get another 64,000 ports.

In Node.js, you can specify which local interface/IP to bind to when making a request:

const http = require('http');

const options = {
  hostname: 'api.target.com',
  localAddress: '192.168.1.101', // Bind to your first secondary IP
  agent: false
};

// Next request could use 192.168.1.102

By cycling through multiple local IP addresses (IP Aliasing), you can effectively scale your outbound connection capacity linearly.

Monitoring Your Exhaustion

Don't wait for your app to crash to know you have a problem. You can monitor how many ports are currently in TIME_WAIT or ESTABLISHED using ss (the modern replacement for netstat).

# Count connections by state
ss -s

Or, to see the raw count of ports currently tied up in the "wait" state:

ss -tan state time-wait | wc -l

If that number is creeping up toward your ip_local_port_range limit, you’re in trouble.

Wrapping Up

Stop blindly increasing your ulimit. It’s like buying a bigger wallet when you’ve actually run out of money.

The next time you see EADDRNOTAVAIL:
1. Check your `ip_local_port_range`. Know your ceiling.
2. Audit your code for Keep-Alive. Are you reusing connections to your DB, Redis, and internal APIs?
3. Check your `TIME_WAIT` count. If it's in the tens of thousands, your app is "leaking" ports.
4. Tweak `tcp_tw_reuse`. Give the kernel permission to be more aggressive with port recovery.

Networking is often the last frontier for application developers. We like to pretend the "cloud" handles the bits and bytes, but the Linux kernel doesn't care about your abstractions. It only cares about the 4-tuple. Respect the limits of the stack, and your Node.js apps will stop falling over at the exact moment they need to perform.