Hacker Remix

WireGuard Performance with a Pi Zero (2019)

79 points by rzk 9 months ago | 71 comments

Sanzig 9 months ago

I have an old Pi 3 installed at my mother-in-law's house running Tailscale (which uses WireGuard as its actual VPN layer). It is connected to my Tailnet along with my Jellyfin server, and I have nginx set up as a reverse proxy to expose the Jellyfin server on the LAN IP of the Pi. This way, she and her sons can access my Jellyfin server as if it were on their LAN - great option for non technical relatives.

This setup has been in place about a year now and just works. The Pi can handle about 50 Mbit bidirectional over WireGuard, which is suffient even for a couple of 4K media streams. I am planning to duplicate this setup at some other relatives' homes.

j-krieger 9 months ago

> This setup has been in place about a year now and just works

For some reason, even with ram-only fs and all common tricks, my Sandisk SD cards keep failing. Do you have any tips?

vinni2 9 months ago

I had this problem with pi 4 after frying several SD cards I found out you can setup read only file system and since then no problems for 3 years now. https://core-electronics.com.au/guides/read-only-raspberry-p...

kstrauser 9 months ago

Boot from a USB SSD instead. I get literally 100x the IOPS over the reasonably fast SD cards I used. Things like apt-get upgrade take seconds instead of many minutes. It’s an entirely different experience.

EasyMark 9 months ago

Get as big of an SD card as you can from a known good company ( I think I have a 256GB card in mine). Turn off as many logging services as you can. You should be able to find several guides on the internet on how to limit writes to the SD card and that combined with a big card with decent wear leveling should last for years, mine has.

NavinF 9 months ago

Is it possible your SD cards are fake? I did full disk writes till destruction and got 1000 cycles with a Sandisk Ultra (their cheapest line of microSD cards)

That should be enough for 10 years under a typical Pi workload like writing and compacting logs.

sweeter 9 months ago

any advice setting something like this up? Also, wouldn't that get expensive?

NavinF 9 months ago

Why would it be expensive?

telgareith 9 months ago

Because an 8gb rpi4 costs close to $160. You can buy a m920q i3 with more compute- and with a similar amount of RAM (Conversion losses, Storage, and then Cooling or RAM(a few watts per 8gb) are the largest power consumers) and it can do a lot more than 50mbit. It might actually use less power than the rpi4. And, it could replace whatever is powering the TV display.

Of course, choose your power supply badly and both those sub 10W machines will be 50W at the wall.

sweeter 9 months ago

I also thought that Tailscale would probably incur some type of charges after using it that much, though Im not super familiar with their free tier policies and how sustainable they are in the long-term.

Sanzig 9 months ago

Tailscale sets up point-to-point WireGuard VPNs and only proxies through their relay servers when they can't establish a direct connection. In my experience that's pretty rare, Tailscale tries a whole bunch of NAT traversal tricks before falling back to relay mode.

Their free tier is pretty generous because it's basically a way for Tailscale to get homelabbers hooked on the product so they'll recommend a corporate plan at work. They even state as much: https://tailscale.com/blog/free-plan

The Pi 3 was essentially free to me because I already had it on a shelf. When I duplicate this setup at some other relatives' homes, I'm planning on using an Orange Pi Zero 3 ($30 CAD, quad core A53, gig of RAM, gigabit Ethernet).

NavinF 9 months ago

They're not proxying your data. That's why there are no usage limits

allset_ 9 months ago

They do proxy your traffic if it can't set up direct connections, and it's still free.

https://tailscale.com/kb/1232/derp-servers

NavinF 9 months ago

Wat.

- You're replying to a thread about someone using a 1GB Pi 3 to stream multiple 4K movies. It's $44 on Amazon including fast shipping. Cheaper on eBay if you can wait 3 days.

- The 8GB Pi 4 is $75 on canakit, not $160.

Anyway if you want more compute (on an edge device? why?), why not grab a AM4 board and CPU for like $80 each? That's 25W at the wall and gives you a ton of flexibility if you later wanna repurpose the machine adding GPUs, NVMe, SAS enclosures, etc

gruez 9 months ago

>The 8GB Pi 4 is $75 on canakit, not $160.

To be fair once you add in shipping, a sd card card, power supply, case/heatsink, and you'll get to around 160.

NavinF 9 months ago

Bizarre. MicroSD cards are $5 on Amazon. I figured everyone has a bunch of spare 5V 2.5A PSUs in the box of wall warts in their garage, but maybe that's a bad assumption. $5 for a brand new PSU and $15 canakit shipping. So it's $100 total if you didn't care at all about cost and bought the most expensive Pi for use as an edge device for no technical reason.

Why would you need a heatsink unless you use a case? Why would you use a case? That price tag is entirely self inflicted

yamrzou 9 months ago

Is it a Pi 3 B+?

whatevermom 9 months ago

Has someone a recommendation for a travel router where I could 1/ setup a WG VPN to encapsulate all my traffic 2/ connect to a Tailscale network?

abound 9 months ago

One of the GL.iNet travel routers [1] would probably work for you. They run OpenWRT (or a thin veneer around it), so you can SSH in and install packages and whatnot. They explicitly advertise Wireguard-based VPN support.

I don't have one of their travel routers, but I have a Flint 2.

[1] https://store.gl-inet.com/collections/travel-ac-router

EQYV 9 months ago

I haven’t managed to get the built in tailscale route-through-exit-node functionality working on my router. Have you / others had success?

abound 9 months ago

Ah I have not. I run a Headscale instance, but my router knows nothing about my Tailnet

sandreas 9 months ago

I'd go for a NanoPI R6S[1]. This thing is a 4 Core beast with USB-C Power Supply support. OpenWRT Support via snapshot, see ToH[2].

If this is too expensive, you could also go for a NanoPi R4S[3], but I wouldn't. The N6S is worth the additional cost.

If you need wifi, there is the R5C[4].

1: https://www.friendlyelec.com/index.php?route=product/product...

2: https://openwrt.org/toh/views/toh_available_16128

3: https://www.friendlyelec.com/index.php?route=product/product...

4: https://www.friendlyelec.com/index.php?route=product/product...

danieldk 9 months ago

The Rockchip in the R6S is very powerful, though depending on what you want to do there may be better options. The R6S doesn't have hardware offloading in OpenWrt. Many Mediatek Filogic SoCs do, so they can do NAT, routing, PPPoE, etc. while the CPU is almost idle. Banana Pi R3/R4 are good options or if you want something that is more of a ready-to-use product and doesn't requite SFP modules, the GL.iNet MT-6000 is really cool: https://www.gl-inet.com/products/gl-mt6000/

Runs their fork of OpenWrt with a user-friendly interface (though LuCi is also available) and you can also flash vanilla OpenWrt. They also have smaller travel models.

Of course if you use stuff that needs to run on the CPU (like Cake), then the R6S will be faster.

sandreas 9 months ago

I personally own a Banana Pi R3 as my main router and it's awesome. Unfortunately, it is pricey and pretty big for a travel router (besides the fact that it must be assembled). The MT6000 is even bigger. And you have to carry an extra power supply.

For traveling I use a Gl.inet Beryl (GL-MT1300), which is nice, but not very powerful. Nowadays I would probably go for a GL-MT3000[1], if there wasn't the NanoPi R5C, which is small, powerful, supports OpenWRT and has Wifi.

As a note: I thought about having Wifi via USB, but the stability and performance of USB-Wifi is nowhere near the integrated / miniPCIe stuff. So if wifi is a requirement, this might be important.

tarruda 9 months ago

I recommend installing tailscale client on your devices instead of carrying an additional device/router

ssl-3 9 months ago

I'll go ahead and install Tailscale on my PS5, then.

Thanks!

throw4950sh06 9 months ago

Why would you need it there? Serious question, would love the use case inspiration.

homebrewer 9 months ago

PlayStation store is not available in many regions, mine included. Not that I personally care, it doesn't make sense to support businesses that treat you like a lesser being.

planetafro 9 months ago

Also remote play is amazing!

sweeter 9 months ago

Chiaki for the SteamDeck is amazing. I love playing Bloodborne on the go.

spr-alex 9 months ago

We (https://supernetworks.org/) have a Tailscale integration https://github.com/spr-networks/spr-tailscale and support Site destinations for devices. For our hardware products one thing we do need is to source a good carrying case for travel.

mech422 9 months ago

Gotta plug my fav's - odroid h2/3/4's ...

Low power, fairly cheap, x86 based, onboard NIC (sometime 2), NVME/Sata and large memory support for lots of containers/etc. Also, low power draw! :-) I've been loving my H2+'s and I got some H4s in I need to find time to play with...

1.) https://ameridroid.com/products/odroid-h4-h4-h4-ultra

2.) https://ameridroid.com/products/odroid-h3 (dual nic)

amatecha 9 months ago

Yeah, GL.iNet GL-AR300M16-Ext is perfect for this purpose, very affordable and compact. You can configure the wireguard client, and then "Block non-VPN traffic" so it allows ONLY connecting through the VPN. Very handy! GL-SFT1200 should be a great option as well, currently the cheapest GL.iNet markets for their "travel AP" line, and you can run Tailscale on it[0]. I'm not sure about the AR300M16.

("Ext" means it comes with external antennas, version without that suffix has internal antenna if you want it to be even more compact)

[0] https://forum.gl-inet.com/t/tutorial-tailscale-on-gl-sf1200-...

fragmede 9 months ago

Damn that one looks pretty good. Are there any with usb-c so I can hook my laptop to it via a usb-c cable and get a usb Ethernet gadget device, and can then carry one fewer cat-5 cable?

xyst 9 months ago

Is the idea of a travel router for the purpose of making sure there are no leaks while using a VPN on a publicly accessible AP?

Client devices -> “travel router” with WG -> public AP

My preferred way is to enable WG on-demand for devices and immediately detect if WiFi or Ethernet is not my home internet.

Client devices (phone, laptop) with WG -> public AP

Or is there some other purpose?

ssl-3 9 months ago

One advantage of a travel router, to me, is convenience. It's pretty great to have my own (portable!) LAN while out and about.

I just show up at the hotel and get my router online.

After configuring that singular device, my other stuff all works together: My Chromecast, my laptop, my smart speaker, whatever gaming system I may have, some ESP32 project or other that I've been tinkering with, or whatever -- I just turn stuff on and it simply works.

With a travel router that additionally uses VPN to tie my travel LAN to my home LAN, then: Whatever other network services I have at home are also available to me on the road.

It can be very transparent.

And that all conspires to mean that I can spend more time doing whatever it is that I feel like doing instead of futzing around with networking.

neurostimulant 9 months ago

Asuswrt-merlin custom firmware can be installed on some asus routers and supports wireguard, among other things.

issafram 9 months ago

I have a Pi 4 and ran Wireguard/PiHole on it for a few years before the SD card died.

I decided to install Ubuntu on a 6 year old Dell XPS computer. I now run Wireguard/PiHole strictly on docker and it is incredibly fast. Changed my settings to auto start the PC after a power loss. I haven't had any downtime for the containers. I'll stick to my custom docker compose file forever.

ycuser2 9 months ago

The only thing is the higher energy consumption.

irunmyownemail 9 months ago

I don't use the expensive Pi devices and like the parent commenter, I use an old laptop with a 4 Gig VM, host Ubuntu, VM Ubuntu and it runs my kube cluster as well as a separate kube cluster on the host itself. If it used much power, my wife would be on me about it. PS I don't use Snap.

doublepg23 9 months ago

Significantly more though? I think people overestimate x86 idle power draw.

ignoramous 9 months ago

WireGuard shouldn't consume energy when idle. Turn off KeepAlive, if your network setup allows for it (on most platforms, the official WireGuard implementation can roam just fine).

abound 9 months ago

I think they meant in case of the Pi vs Dell XPS

fnord77 9 months ago

Does the XPS use a lot more power than the pi 4?

EasyMark 9 months ago

Of course it does. It’s probably still less than a few dollars a month

chao- 9 months ago

Would you share said compose file?

stavros 9 months ago

I can't speak to the Compose file itself, but I use Compose to run stuff myself on an intel NUC and it has been amazing. Orders of magnitude faster than a Pi, super stable, tiny, I just love it.

I even wrote a utility to manage the bunch of Compose files via git and automatically update them when I push changes to the repo: https://harbormaster.readthedocs.io/en/latest/

disqard 9 months ago

Thank You For Making And Sharing :D

fnord77 9 months ago

> I’d say that if you’re planning on using WireGuard on an iOS device with the On-Demand Activation for untrusted wi-fi networks when away from the house, this should get the job done to protect you on public wi-fi networks. If the goal is permanent, high throughput usage, I would recommend a more powerful box to run WireGuard.

A zoom meeting on a phone is pretty high throughput...

PhilipRoman 9 months ago

Is it really? For personal use I find that anything except file transfers uses a tiny amount of bandwidth (few MBit/s at most). That includes stuff like video calls, remote desktop, youtube, etc.

EasyMark 9 months ago

Not when most households are getting 30Mbps up and 300Mbps down or more. Now several at once would strain it for sure.

yamrzou 9 months ago

Does anyone have suggestions for the smallest physical device that can function as a WireGuard server or a Tailscale exit node with decent performance?

toomuchtodo 9 months ago

I have had great luck with https://www.gl-inet.com/ travel routers as line speed Wireguard endpoints. Works on fiber and StarLink equally well.

aborsy 9 months ago

They have also Tailscale plug-in. You have to trust the company out of China or HK, though.

dbrueck 9 months ago

I agree with this recommendation - they work great with Wireguard. And if you're travelling, some of the features like handling captive portals are handy.

yamrzou 9 months ago

They are good wireguard clients but not servers

zekica 9 months ago

What's the difference?

yamrzou 9 months ago

On GL.iNet website they state: "OpenVPN and WireGuard speeds will be slower when running the device as a server. Results above are in client mode."

dudus 9 months ago

The Lenovo Thinkcentre M series tiny or a HP mini are the sweet spot for me.

For less than $200 you can get a used one with 16GB of RAM and a fast SSD.

For home servers I want low power usage and reliability. Mine idle at 5W running proxmox.

caconym_ 9 months ago

This explicitly doesn't answer your question as written, but just in case it's relevant to you anyway: you can run something like pfSense in a VM on a server or really any machine you have available on the network where you want an exit node. At least on Linux, the software networking support is good enough to make such a VM appear as just another machine on the network the VM host is connected to.

My primary home router is a pfSense VM set up as a Wireguard peer for tunneling in from various other devices and locations, and I'm very happy with it.

KaiserPro 9 months ago

Probably something like an n100 based "NUC" type deal. Its has loads of float performance and is much better suited to being a "server" than a pi (much as I love the pi)

zamadatix 9 months ago

If the goal is smallest VPN box instead of best for the price server then the float performance doesn't really matter much and both are probably overkill -> too large. Both the n100 and the pi 5 can reach multiple gbps of wireguard throughput, whatever you can get in the smaller total form factor is more ideal than ridiculous throughput.

A table of devices and wg speeds can be found here https://forum.openwrt.org/t/a-wireguard-comparison-db/187586. There are plenty of interesting tiny options, particularly if you don't need a full gig.

poisonborz 9 months ago

GLiNet AR300M Travel router. I don't think you could make a smaller one even going DIY (with a case, that is). Perf is 50mb with Wireguard officially.

Hamuko 9 months ago

I'm currently using my Unifi Cloud Gateway Ultra router as a Wireguard server for my home network and it's at least somewhat compact with good performance. Before that I used to have a Dell WYSE 3040 that's also quite compact but maybe a bit less so on the performance side.

petepete 9 months ago

Maybe not the absolute smallest but Unifi cloud gateways are very small.

https://ui.com/us/en/cloud-gateways/compact

twic 9 months ago

I run a WireGuard server on my wireless router. The router itself is not tiny, the size of a two-inch-thick trade paperback. But the marginal size of the WireGuard device is zero, because i need the router anyway.

flemhans 9 months ago

Anyone got any opinions on max number of tunnels? How does performance degrade as you have thousands of simultaneous tunnels?

TomK32 9 months ago

This from 2018 says the max number per interface is 2^20 for the kernel module but it can be raised. https://news.ycombinator.com/item?id=17093621

ThePowerOfFuet 9 months ago

Saved you a click:

>As expected, the speed is around 90 megabits per second, as the Pi Zero has a USB 2.0 OTG port, and I’m using a 100mb ethernet adapter for it.

ZeKK14 9 months ago

That's the result without wireguard. With wireguard:

> depending on the use case for a Pi Zero WireGuard server, it could get the job done with ~30-40 megabits per second speed capabilities.

ThePowerOfFuet 9 months ago

Right you are! Was not clear at all at first glance.