Traffic Shaping Processes in Linux

I live in Australia, and while this is on the whole a lovely country, our internet sucks. Check out my awesome upload speed:

Now while this does not cause me grief for the most part, I recently attempted to upload a large number of photos to Google Drive. And as soon as I set Google’s Backup and Sync App (in Windows) to run in the background - Me and everyone else in the house could no longer access the internet. The sync app was clogging up the entire upload bandwidth, so that sometimes even a measly DNS request couldn’t get through.

This should be an easy problem to fix with sensible QoS settings on a router, but what if you did not have access to a router, or you’re in a network you don’t control?

In linux, it turns out there is a neat way to control the traffic of a process. It’s called - tc, and the method is described perfectly in this stack exchange answer. We’ll explain how this works, and wrap it up in a useful script to make it more generic.

First, we create a net_cls cgroups instance. This is a Network classifier cgroup:

The Network classifier cgroup provides an interface to tag network packets with a class identifier (classid). Source

We will call ours slow, and the command is:

# Create a net_cls cgroup
cgcreate -g net_cls:slow

Next, we create a new class id to associate with this cgroup.

Creating a net_cls cgroups instance creates a net_cls.classid file. This net_cls.classid value is initialized to 0.

You can write hexadecimal values to net_cls.classid; the format for these values is 0xAAAABBBB; AAAA is the major handle number and BBBB is the minor handle number. Source

We choose to set our classid as 1:1 or 0x10001:

# Set the class id for the cgroup
echo 0x10001 > /sys/fs/cgroup/net_cls/slow/net_cls.classid

Now we can classify all packets originating from a PID to the cgroup instance:

# Classify packets from pid into cgroup
cgclassify -g net_cls:slow <PID>

And finally, we configure tc to perform the rate limiting. We’ll explain the commands one by one:

# This modifies the scheduler (queuing discipline), applies the rule to the eth0 interface, outbound (egress) traffic only (aka root)
# The handle is the major number associated to the class
# htb is the type of shaping algorithm (hierarchical token bucket)
tc qdisc add dev eth0 root handle 1: htb

Next we add a filter to the qdisc to associate it to a cgroup:

tc filter add dev eth0 parent 1: handle 1: cgroup

And finally, we add a class to the previously created filter, and rate limit it to any rate you wish:

tc class add dev eth0 parent 1: classid 1:1 htb rate 1mbps

More information about tc can be found in the man page here

We can wrap all of this up in a small script, which can rate limit any command you wish. For example, to rate limit an rclone to google drive we only need to specify the rate to throttle and the interface, before the command itself:

./rate_limit.sh 10kbps eth0 rclone copy ~/Pictures/ google-photos:/album/testAlbum --update --checksum --transfers 1 --stats 10s -vvv

A copy of the script can be found here