remix logo

Hacker Remix

Intel's Tofino P4 Software Is Now Open Source

203 points by ram_rattle 6 months ago | 76 comments

dfex 6 months ago

Am I mistaken or has Intel pretty much shelved the Tofino switching hardware that supports P4 in the first place?

I seem to recall Oxide having to switch suppliers over this?

wmf 6 months ago

Yes, Intel canceled Tofino two years ago.

I'm not complaining but it's weird that they're open sourcing the SDK now. Maybe it's to support Mount Evans.

dolmen 6 months ago

Releasing dead software that is used only to support dead hardware?

spookie 6 months ago

At least you can still make use of that hardware, many companies should take note of this. You can make hardware and software that doesn't die once someone pulls the plug on the other side.

Still, of course it would've been better to have released it sooner.

snizovtsev 6 months ago

I think half of Tofino's complexity was in their compiler. So it may inspire new hardware vendors to reuse it in some contexts.

robertlagrant 6 months ago

That's fair enough.

rising-sky 6 months ago

This was the post by Bryan Cantrill of Oxide on the Tofino saga with Intel

https://bcantrill.dtrace.org/2024/12/08/why-gelsinger-was-wr...

ggm 6 months ago

I also was told tofino was looking EOL. Like NUC, dropped from some C suites KPI set and longterm roadmap.

I'd love to be wrong, this is just what people said.

bayindirh 6 months ago

However, NUC form factor is more lively than ever before. Maybe someone can grab the language and run for something? Miktrotik guys come into my mind.

luckman212 6 months ago

What's the best "NUC-like" product on the market now? I know ASUS has their lineup of "NUC" that was spun off from Intel. I have used MSI's "Cubi" which was a pretty nice kit. And I know everyone drools over Minisforum stuff (which is expensive but very nicely designed). Any other notable ones?

bayindirh 6 months ago

Depends on what you want. Currently my focus is N100 based systems to use as lightweight personal computers and small servers (if you can find fanless versions).

I have BeeLink S12 Pro, which I post this comment from, and I like it. However I/O wise, it's not the fastest (x1 NVMe + 2.5" SATA SSD). I also have a GMKTec G3, which has a x2 NVMe and M.2 SATA Slot. I have recently seen a "Minix Z100" fanless, which has x4 NVMe but no SATA, however it has 2.5gbE. Lastly, my parents are using a similar N95 system on Windows 11, and it works pretty nicely for them. All of these systems support 32GB of single channel RAM, but they're plenty fast for simple jobs or simple desktop systems which you won't try to conquer the world.

If you have no power/heat/noise/money upper limit, you can get them in more packed configurations, but that's not what I need, actually.

doctorpangloss 6 months ago

It’s complicated. The NUCs had a lot of inevitable catastrophic hardware failures, like NIC ports that would break. Their problem is they were not good.

UltraSane 6 months ago

How is a ethernet port breaking "inevitable"?

doctorpangloss 6 months ago

100% of the Broadwell NUCs, AFAIK, eventually have the problem, but many were trashed before they did. These issues persisted for years, such as with 10th and 12th generation NUCs. Search "proxmox nuc NIC not working".

Twirrim 6 months ago

Broadwell is gonna broadwell. I won't ever trust their NICs ever again. They've been way too much pain with them both personally and professionally.

Their firmware is crap quality, and their bugs are just absolutely astoundingly bad.

At one job we literally had to fix their firmware for them after several months of back-and-forth, engineers spotted the absurdly obvious bug in minutes of seeing their code.

happycube 6 months ago

NUC somewhat avoided the google^Wintel graveyard - they sold at least the branding to Asus.

p_l 6 months ago

NUCs are shifting completely to ASUS who is going to continue working on them and there are some long term commitments (there are industrial variants for example)

dvtkrlbs 6 months ago

They didnt switch supplier iirc they are still using Tofino since it is is still a capable hardware and they see it being useful for years to come

danpalmer 6 months ago

They spoke on their podcast (I think it was there?) about ditching Tofino for the next generation of the Oxide computer. So it sounds like the current model will always ship with Tofino, but due to no future product development they won't use it again in a new machine. It sounded like they had just secured a replacement for the future but I can't remember who it was.

bcantrill 6 months ago

We've talked about it a bunch, most recently when talking about Intel after Gelsinger.[0] I went into more detail on Intel's total mishandling of Tofino in my blog entry describing why Gelsinger was the wrong choice to lead Intel in 2021.[1]

As you might imagine, this move from Intel is something that we at Oxide have advocated for strenuously -- and it is a tremendous tribute to the former Tofino team at Intel that this got done. As I hope I made clear in my blog entry: the folks working on Tofino at Intel have been great to work with; they deserved much better than their (former) executive leadership.

[0] https://oxide-and-friends.transistor.fm/episodes/intel-after...

[1] https://bcantrill.dtrace.org/2024/12/08/why-gelsinger-was-wr...

danpalmer 6 months ago

Thanks for the context, your respect for the Tofino team certainly came through in the podcast.

dfex 6 months ago

Thanks Brian [0] was a great podcast and got me through a large lawn mowing!

mikeyhew 6 months ago

Does this move to open source P4 have any impact on Oxide at all, or is it too little too late?

bcantrill 6 months ago

We definitely advocated for it (at all stages of our interaction with Tofino -- but especially strenuously after the part was killed), and it does have impact on us in that it was one of the few parts of our stack that we couldn't open (and now we can -- stay tuned there). We remain big believers in P4 and want to see P4 compilers for other switching silicon, so getting an open is a big step in that direction. I would also say that what Intel did here beat our expectations, and includes many aspects of their software that we didn't think they would make available. So even though we will be moving beyond Tofino, getting their software open source is great and we're very supportive -- and if anything, it just sharpens for us what a poor strategic decision it was to kill Tofino.

mmmBacon 6 months ago

P4 has more or less gone nowhere. Tofino was a full generation behind and didn’t make sense. P4 was compelling because people thought they’d solve the Elephant flow problem with traffic engineering in P4 but the resources to actually do this at scale never materialized for many reasons.

vriendspookstem 6 months ago

P4 as a language is still used on some programmable NICs for describing packet parsing dataplanes and exists as an HDL. Unfortunately the SDKs are not always exposed to small scale companies or universities like Intel did with Tofino.

yusyusyus 6 months ago

ehh scream SDN 5 times. kinda miss the 2010’s now.

cisco silicon one uses p4 fwiw. internal development though, but the language makes sense for what the things are.

0xNOTVALID 6 months ago

do they expose the p4 functionality externally? ive heard this from them but never actually seen the proof - it seems like vaporware

wmf 6 months ago

I don't think they would lie but they just don't release the compiler. Maybe they give it to Meta.

vriendspookstem 6 months ago

yes,they might provide it to hyperscalers or allows them to customize some of the packet parsing pipeline for these companies.

lifeisstillgood 6 months ago

I’m impressed by the political acumen it takes to get a Corp to release code as OSS. My career has seen at least two chunks of work that would have made great OSS (ie potentially useful outside of the single company) but Incoukd not get past the final hurdle

And they would have been nice CV boosters as well (my real motivation!)

dolmen 6 months ago

Well. It looks like abandonning software to the community.

From the "P4 workflow" described at https://p4.org/ I see mentions of compiling to x86, but no mention of ARM, and no mention of BPF. So, as someone who discover it, I wonder if this project is still relevant in 2025.

enragedcacti 6 months ago

I'm not familiar with P4 but with some quick digging it seems like the compiler does support eBPF as a target. Additionally, I don't think the compiler outputs x86 directly at all, and instead includes targets like DPDK which supports ARM in addition to x86.

https://github.com/p4lang/p4c

FuriouslyAdrift 6 months ago

There's a bunch of network operating systems that use P4 (Arista EOS, Cisco NX-OS, IPinfusion OCNOS, Microsoft SONiC, etc)

trimethylpurine 6 months ago

Is x86 dead for wire powered devices?

Regardless, OSS is probably the best way to get it onto other architectures.

neuroelectron 6 months ago

Basically, now you have to document x86. xD

baq 6 months ago

It helps that Intel has been contributing to OSS since forever, they have good internal processes established and some orgs develop/contribute to OSS pretty much exclusively.

bayindirh 6 months ago

I think the biggest contributor to these processes was going open source in their graphics drivers.

They went from "we have tons of 3rd party IP in these!" to, "you don't need to download anything, it's in kernel mainline now" in a generation and they're off to the races after that.

Maybe their Ethernet drivers were open before that, I don't remember but, video drivers made them pass a threshold in maturity IMHO.

spookie 6 months ago

Intel embraced Linux when they bet big in datacenters in early 2002. The momentum they got from there, with Linux dominating the sector, made them realize how much of a benefit it is to have a free software stack. With Linux they were able to go head to head with the incumbents of the day.

Linux and x86 became unbeatable in the space for 20 years.

They have known how important it is. They won't forget.

bayindirh 6 months ago

Yes, however, they first made their hardware standard compliant rather than making their driver open source.

The open source part came later, starting with CPU and chipset support, then Ethernet, then GPUs IIRC.

The biggest and sweetest side-effect is Desktop/Personal use Linux support as long as the hardware doesn't do anything janky, or too janky.

spookie 6 months ago

Fair enough

bombcar 6 months ago

The ethernet ones were big for awhile, I can remember when you wanted to go out of your way to get something that worked with the e1000 driver if you wanted reliable, usable gigabit.

fidotron 6 months ago

> they have good internal processes established

There are a good number of people that would LOL at this statement, myself included.

Maybe they have such processes now, because at one point . . . Well “mistakes were made”.

kanwisher 6 months ago

I had one piece that made it open source (with blessing of 20 committees), then someone that didn't like me ran it back up the flag poll and pulled it back next day

lifeisstillgood 6 months ago

Oooff!

antithesis-nl 6 months ago

As far as I can tell, Software Defined Networking (which this is about: "P4 is a domain-specific language for network devices") by now is pretty much a decade-old promise that never materialized. I'd still love to be wrong though!

So, let's take the next paragraph: "Before P4, vendors had total control over the functionality supported in the network (...) controlled the rollout of new features (e.g., VXLAN), and rollouts took years"

Anyone has a pointer to any actually available hardware capable of L2 and L3 packet processing where I could have implemented VXLAN in, say, weeks using P4? Again, as far as I can tell, it's all either killed-off-a-long-time-ago, "contact us" vaporware, or exotic 40/100-Gb-only Top-o-Rack gear, and even for those, there is nary an "add to cart" button in sight...

wmf 6 months ago

P4 is used by Cisco Silicon One, Xsight, AMD Pensando, Intel Mount Evans, etc. "Contact us" isn't the same thing as vaporware; the Pensando SSDK and Intel ES2K SDE definitely exist for example. I realize it sucks when things aren't available to hobbyists but it's a mistake to pretend it's fake.

P4 is really only needed in data center networks because slower campus/home networks can usually get away with software processing and their lower prices probably can't support the R&D of a programmable architecture.

kuon 6 months ago

I would be curious as well. Every time we try something "software defined" the drawbacks are major, cost goes up, stability goes down and most importantly, bandwidth goes down by a factor 10. The only software defined networking gear we use is OpenBSD, to do some complex routing, but we cannot go above 5GB/s.

newsclues 6 months ago

Seems to be used in the military radio space.

eqvinox 6 months ago

> This bold move from Intel to open-source the Tofino P4 software is more than just a licensing decision; it’s a call to action for the global developer community.

Nah. 5 years ago this would've been bold. Now it's ridding yourself of the baggage of an almost-dead platform that you're about to make fully dead.

Still appreciate getting the tooling as FOSS rather than just terminating it, but let's not go for delusions here.

unixhero 6 months ago

What are they making instead?

eqvinox 6 months ago

To my knowledge: nothing. Intel is exiting the network switch silicon business¹. Broadcom and nVidia are dominating (different parts of) the market; Marvell and Microchip are fighting for the scraps.

The only "cool" player is Microchip, who have been providing full datasheets, register maps, and open sourcing their drivers for years now. But I'm under no illusions they're doing this out of the goodness of their heart, they're doing it because it's one of very few competetive advantages available to them.

(Which is perfectly fine! FOSS drivers are a great competetive advantage! It's not working super well sadly :/ — but part of the problem here is Broadcom's anticompetetive behavior. To my knowledge, any switch OEM producing Broadcom-based gear will get their NDAs and silicon access revoked if they so much as dream about making devices with non-Broadcom silicon.)

¹ Intel has already exited this business some while ago, they only bought Fulcrum Micro to get better NICs basically since every NIC is nowadays also a switch. Tofino was always a "special beast", not quite competing against e.g. Qumran or Trio. Tofino is (was?) better thought of as special purpose FPGA…

kogepathic 6 months ago

> To my knowledge, any switch OEM producing Broadcom-based gear will get their NDAs and silicon access revoked if they so much as dream about making devices with non-Broadcom silicon.

Cisco Meraki did; their low end switches are Marvell and their "high end" switches (MS420, MS425, MS450, MS350, MS355) were all Broadcom based. Were because about a year ago they announced the End of Sale of all Broadcom based switches.

Everything above the low end stuff is now Cisco Catalyst. (Although one can argue everything from Meraki is low end apart from their prices)

> Marvell and Microchip are fighting for the scraps

Realtek also. Lots of smaller L2 managed switches based on the RTL93xx series. [1]

But I am not seriously comparing Realtek to Tofino, that's like comparing Hot wheels to the actual car.

[1] https://svanheule.net/switches/

UltraSane 6 months ago

"Although one can argue everything from Meraki is low end apart from their prices"

This is hilariously accurate.

p_l 6 months ago

AMD ended up entering the market to some extent - big powerful software-modifiable NIC chips that can also serve as switches from my understanding.

But like Tofino, it's mostly stuff that is "behind the curtain"at hyperscalers or deep inside closed box switches

nsteel 6 months ago

> To my knowledge, any switch OEM producing Broadcom-based gear will get their NDAs and silicon access revoked if they so much as dream about making devices with non-Broadcom silicon.

Nokia use Broadcom silicon for low-end, in-house for the rest.

dist1ll 6 months ago

Hopefully they don't axe their NICs. At least Intel provides really good open datasheets for their e800 cards, can't say the same about other vendors.

touisteur 6 months ago

The absence of anything >200G isn't reassuring there...

14 6 months ago

Not much to add technically but I am very curious who named it Tofino. Tofino is a city by where I live. It is a beautiful location on the Pacific Ocean on Vancouver Island. A favorite destination for our PM Justin Trudeau. Definitely one of the top 100 beautiful places in the world. When I go there I feel stress leave my body and just enjoy the sounds of the waves crashing and the feeling of the sand on my feet melt any worries I may have. Love it there.

jnf27 6 months ago

The founding CEO of Barefoot Networks was big surfer. They used famous surfing spots as the internal names of their ASICs.

bnchrch 6 months ago

Same here! Im living over in Comox and spend a lot of time in Tofino / Ucluelet. Best part of all of Canada in my opinion.

I particularly enjoy the winter trips to SF to show my colleagues pictures of the latest cold water surf adventure.

14 6 months ago

Nice we are basically neighbors you have to drive past my house to get to Tofino. If you are ever passing by always like making new like minded friends.

gnabgib 6 months ago

Related P4: open-source programming language for high-performance packet switching (98 points, 2016, 19 comments) https://news.ycombinator.com/item?id=11903478

RhettS 6 months ago

There's more to this than meets the eye. We are building the world's first open, softwre only, intelligent, content-aware switches. Dual protocol stacks, our SPAN-AI and TCP/IP. This saves us having to develop the TCP/IP stack and a bunch of router code. Brilliant. It also opens up the market for Intel X86 based "Smart Switches" i.e. X86 servers with SmartNICs which is our first platform. Again very smart. This may the first glimmerings of the new Intel. We hope so! Rhett, founder CTO of https://gtsystems.io/

deivid 6 months ago

After having used & automated configuration of "traditional" switching/routing devices for years (Cisco , Arista, Juniper), I can't wait for P4 devices to take over

fishstock25 6 months ago

I believe that Arista has (had?) P4 devices in their lineup, but after Intel's sunsetting of that platform, they probably axed them.

UltraSane 6 months ago

You know all those random python automation scripts network engineers use to configure devices? Now imagine them actually running on device controlling packet and frame flows. Does that sound like it would be fun to troubleshoot?

wmf 6 months ago

What do you want to use P4 for?

dfex 6 months ago

The dream is you never have to rely on buggy vendor software again.

The reality is, you end up with a complex stack with no or homegrown documentation that requires experienced engineers to operate and maintain.

In some environments, that's a perfectly fair trade-off. In most, it isn't.

deivid 6 months ago

The APIs these vendors provide are a joke. A bunch of functionality can only be accessed via scripting interactive CLI commands. Some API endpoints cause short downtime / unexpected behavior (eg: by deleting the routing table and adding all entries back 1 by 1), while the on-device commands do not.

And guess what, the switch may decide to print informational or environmental messages interleaved with the command output, because the commands were meant to be run by a human. Good luck knowing if your state-altering command succeeded when you receive broken JSON.

I ended up regex-removing known environmental messages from command outputs..

eqvinox 6 months ago

Improvement on this is more likely to come from switchdev than from Tofino & P4. (though these don't necessarily contradict each other)

You can already run plain Debian on a Mellanox Spectrum device, treat it like a Linux software router, and by the power of magic your routes get pushed into hardware. (Source: device on my table to my right :D) Microchip's SparX-5 should be similar though I don't have one of those to test.

dfex 6 months ago

That was certainly the case 15 years ago.

The only switching/routing vendor with an API worth a damn was Juniper - in fact they were the only vendor who was applying CLI changes to the box via their own API, the Way It Should Be Done™.

These days you are spoilt for choice (and price point) with the likes of Arista, HPe (AOS-CX), Cisco (NX-OS, IOS-XR) and plenty of others entering the space.

Vote with your wallet!

purpleidea 6 months ago

Unless there are devices that I could buy (48 port 1U style switches) and until this supports integrating with something like switchdev ( https://docs.kernel.org/networking/switchdev.html ) then this is too little too late. I hope I'm wrong though.

does anyone know if someone can make this switchdev friendly?

wmf 6 months ago

There have always been Tofino switches available to buy but they were priced for the data center not the home. You can probably find them on eBay now that they're old enough.

Writing a switchdev driver is an enormous amount of work so it only happens when someone (usually the chip vendor) pays for it. Any code open-sourced by Intel probably won't meet Linux kernel standards so it would have to be completely rewritten (like the tg3 driver from the old days).

pinoy420 6 months ago

This is not Perforce.

Abismith 6 months ago

[dead]

unit149 6 months ago

[dead]

kissgyorgy 6 months ago

[flagged]