Rework bandwidth speeds document

- Re-do all measurements
- Add Wireguard
- Use the same speedtest servers
- Add more conclusions
Quentin McGaw (desktop)
2021-09-16 13:09:47 -04:00
parent 1fccf45c83
commit 69741f8d64
+50 -19
@@ -8,37 +8,68 @@ I ran multiple tests using different clients, servers and regions to clarify all
- The further away the VPN server is, the slower the bandwidth will be
- Running Docker on a VM/Windows/OSX brings a serious bottleneck due to virtualization
- The VPN protocol (`VPN_TYPE=openvpn` or `VPN_TYPE=wireguard`) have very little impact. Wireguard does have a more significant impact on low power devices though (+15% speeds).
- The VPN protocol (`VPN_TYPE=openvpn` or `VPN_TYPE=wireguard`):
- `wireguard` is almost always slower than `openvpn`, maybe due to server load.
- `wireguard` does have a more significant impact on low power devices (+15% speeds)
- VPN server for OpenVPN and Wireguard are often not the same even for the same location, and so have different server load
- The speedtest server can give very different results over time
- The difference of results from one speedtest server to another (even in the same city) can be largely different
- Bandwidth fluctuates during the day due to other VPN clients on a particular server
- Some VPN providers may:
- throttle down the download bandwidth when you run a speedtest (e.g. Mullvad)
- allocate you less bandwidth if you don't use their proprietary software
- Docker might be a bandwidth bottleneck
### Investigations
#### OpenVPN
- Only when the VPN server is far away, Gluetun's OpenVPN is x4 slower than native openvpn. This might be due to OpenVPN being built for `musl` or due to other libraries such as `openssl` built for `musl` (Alpine).
- Running OpenVPN **or** Wireguard natively on Windows gives higher bandwidths than when using a container on a Linux host. This is even stranger since Wireguard is supported, since its implementation is the same one for both Gluetun and the Wireguard Windows client. My only explanation is that Docker causes a bottleneck. **This should be investigated once gluetun can be run out of a container as a binary**.
## Testing
### Setup
- I'm based in Montreal Canada
- I am using Mullvad
- Testing on the following CPUs:
- Ryzen 5900x running a Windows host
- Ryzen 2600x running an Arch Linux host
- My bandwidth without VPN is around 1Gbps up and down.
- Gluetun `v3.24.0` is used
- The Mullvad VPN service provider is used
- We test on multiple machines:
Summary table:
| Nickname | Host OS | Docker | CPU | CPU arch | CPU virtual cores |
| --- | --- | --- | --- | --- | --- |
| `windows` | Windows | Docker desktop on WSL2 | AMD 5900x | `amd64` | `24` |
| `arch` | Arch Linux | Docker | AMD 2600x | `amd64` | `12` |
| `rpi4` | Raspbian 32 bit | Docker | Raspberry Pi 4 A72 | `arm64` | `4` |
| VPN server | Openvpn host | Speedtest host | Openvpn cores | Download average |
| --- | --- | --- | --- | --- |
| Paris | Gluetun on Linux | Gluetun container | 12 | **50** |
| Paris | Gluetun on Windows | Gluetun container | 6 | 55 |
| Paris | Openvpn on Windows | Windows | 24 | 250 |
| Paris | Openvpn on Windows | Alpine container | 24 | 192 |
| Montreal | Gluetun on Linux | Gluetun container | 12 | **500** (wtf?) |
| Montreal | Gluetun on Windows | Gluetun container | 6 | 107 |
| Montreal | Openvpn on Windows | Windows | 24 | 355 |
| Montreal | Openvpn on Windows | Alpine container | 24 | 375 |
- We use `speedtest-cli` to test the bandwidth on all speedtest hosts with:
```sh
speedtest-cli --no-upload --server <server-id>
```
On Windows I use: `speedtest -s <server-id>`
- We use the following Speedtest servers
| Nickname | ID | Name | Distance to VPN server |
| --- | --- | --- | --- |
| `amsterdam` | `26425` | ExtraIP (Amersfoort, Netherlands) | 41.77 km |
| `montreal` | `4393` | TELUS (Montreal, QC, Canada) | 0.35 km |
### Results
*Note: `native` client host means we run the Windows program without Docker.*
| VPN server | Speedtest server | Machine | VPN client host | Speedtest host | Protocol | Highest of 3 download Mbps |
| --- | --- | --- | --- | --- | --- | --- |
| `nl-ams-001` | `amsterdam` | `arch` | `gluetun` | `gluetun` | `openvpn` | `110` |
| `nl1-wireguard` | `amsterdam` | `arch` | `gluetun` | `gluetun` | `wireguard` | `85` |
| `nl-ams-001` | `amsterdam` | `windows` | `gluetun` | `gluetun` | `openvpn` | `54` |
| `nl1-wireguard` | `amsterdam` | `windows` | `gluetun` | `gluetun` | `wireguard` | `42` |
| `nl-ams-001` | `amsterdam` | `windows` | native | native | `openvpn` | `206` |
| `nl1-wireguard` | `amsterdam` | `windows` | native | native | `wireguard` | `115` |
| `ca-mtr-101` | `montreal` | `arch` | `gluetun` | `gluetun` | `openvpn` | `310` |
| `ca10-wireguard` | `montreal` | `arch` | `gluetun` | `gluetun` | `wireguard` | `310` |
| `ca-mtr-101` | `montreal` | `windows` | `gluetun` | `gluetun` | `openvpn` | `194` |
| `ca10-wireguard` | `montreal` | `windows` | `gluetun` | `gluetun` | `wireguard` | `210` |
| `ca-mtr-101` | `montreal` | `windows` | native | native | `openvpn` | `530` |
| `ca10-wireguard` | `montreal` | `windows` | native | native | `wireguard` | `512` |