remix logo

Hacker Remix

Physically Based Rendering: From Theory to Implementation

214 points by ahamez 5 days ago | 64 comments

crispyambulance 1 day ago

Every time I see stuff like this it makes me think about optical design software.

There are applications (Zemax, for example) that are used to design optical systems (lens arrangements for cameras, etc). These applications are eye-wateringly expensive-- like similar in pricing to top-class EDA software licenses.

With the abundance GPU's and modern UI's, I wonder how much work would be involved for someone to make optical design software that blows away the old tools. It would be ray-tracing, but with interesting complications like accounting for polarization, diffraction, scattering, fluorescence, media effects beyond refraction like like birefringence and stuff like Kerr and Pockels, etc.

hakonjdjohnsen 1 day ago

This, very much this!

I do research in a subfield of optics called nonimaging optics (optics for energy transfer, e.g. solar concentrators or lighting systems). We typically use these optical design applications, and your observations are absolutely correct. Make some optical design software that uses GPUs for raytracing, reverse-mode autodiff for optimization, sprinkle in some other modern techniques you may blow these older tools out of the water.

I am hoping to be able to get some projects going in this direction (feel free to reach out if anyone are interested).

PS: I help organize an academic conference my subfield of optics. We run a design competition this year [1,2]. Would be super cool if someone submits a design that they made by drawing inspiration from modern computer graphics tools (maybe using Mitsuba 3, by one of the authors of this book?), instead of using our classical applications in the field.

[1] https://news.ycombinator.com/item?id=42609892

[2] https://nonimaging-conference.org/competition-2025/upload/

bradrn 1 day ago

> I am hoping to be able to get some projects going in this direction (feel free to reach out if anyone are interested).

This does sound interesting! I’ve just finished a Masters degree, also in non-imaging optics (in my case oceanographic lidar systems). I have experience in raytracing for optical simulation, though not quite in the same sense as optical design software. How should I contact you to learn more?

hakonjdjohnsen 23 hours ago

Interesting! I added an email address to my profile now

bradrn 22 hours ago

Great! I’ll send you an email now.

accurrent 1 day ago

hakonjdjohnsen 23 hours ago

Yes, exactly. I have not looked at Mitsuba 2, but Mitsuba 3 is absolutely along these lines. It is just starting to be picked up by some of the nonimaging/illumination community, e.g. there was a paper last year from Aurele Adam's group at TU Delft where they used it for optimizing a "magic window" [1]. Some tradeoffs and constraints are a bit different when doing optical design versus doing (inverse) rendering, but it definitely shows what is possible.

[1] https://doi.org/10.1364/OE.515422

roflmaostc 20 hours ago

Shameless plug, we use Mitsuba 3/Dr.JIT for image optimization around volumetric 3D printing https://github.com/rgl-epfl/drtvam

pjmlp 17 hours ago

It looks quite interesting, especially the part of scripting everything in Python with a JIT, instead of the traditional having to do everything in either C or C++.

Looking forward to some weekend paper reading.

hakonjdjohnsen 19 hours ago

Looks really cool! I look forward to reading your paper. Do you know if a recording of the talk is/will be posted somewhere?

roflmaostc 13 hours ago

We presented this work at SIGGRAPH ASIA 2024. But I think they do not record it?

Maybe in some time we also do an online workshop about it.

accurrent 20 hours ago

I dont know much about Optical engineering, but this sounds super exciting. I think I meant to point to mitsuba 3, not 2.

ska 13 hours ago

This is one example of an area where economic incentives make it difficult to shift.

  - There aren't that many people willing to pay for such software, but those that do *really* need it, and will pay quite a bit (passing that cost on of course). 
 
  - The technical domain knowledge needed to do it properly is a barrier to many
 
  - It needs to be pretty robust
As a result, you end up with a small handful of players who provide it. They have little incentive to modernize, and the opportunity cost for a new player high enough to chase most of them off to other avenues.

I think the main way this changes is when someone has already spend the money in an adjacent area, and realized "huh, with a little effort here we could probably eat X's lunch"

Beyond that you at most get toy systems from enthusiasts and grad students (same group?) ...

Q6T46nT668w6i3m 1 day ago

You’d be surprised! Everywhere I’ve worked, academic or industry, typically writes their own simulation software. Sometimes it’s entirely handwritten (i.e., end-to-end, preprocessing to simulation to evaluation), sometimes it’ll leverage a pre-existing open source package. I imagine this will become more and more common if, for no other reason, you can’t back-propagate an OpticStudio project and open source automatic differentiation packages are unbeatable.

lcrs 1 day ago

If you're interested in the equivalent of "backprop through zemax" there are a few projects going on to jointly optimize optical designs with the image processing, e.g. check out: https://vccimaging.org/Publications/Wang2022DiffOptics/

lcrs 1 day ago

I've been working on something similar, although I'm more interested in replicating the effects of existing lenses than designing new ones: https://x.com/dearlensform/status/1858229457430962318

PBRT 3rd edition actually has a great section on the topic but it's one of the parts that wasn't implemented for the GPU (by the authors, anyway): https://pbr-book.org/3ed-2018/Camera_Models/Realistic_Camera...

timeforcomputer 1 day ago

I love this book so much. The literate programming style I think inspired from Knuth's cweb, great writing, beautiful high-quality physical book worth buying but also free access to knowledge. The literate-programming style means you are almost immediately applying theory to a practical system, I keep having to take breaks to learn outside math/physics though, but it is self-contained in principle I think.

losvedir 1 day ago

All right, off topic but I've seen this a bunch lately and the term just really irritates my brain for some reason. What's its origin? "[adverb] based" just feels so wrong to me. Shouldn't that be a noun: "Evidence-based medicine", "values-based", "faith-based", etc. Does "physically based" bother anyone else?

corysama 14 hours ago

It is a bit of a silly term. It was made mostly to contrast against the somewhat more adhoc methods that preceded it. The new technique was parameterized more around physical properties where the older ones were more about visual descriptions.

This paper from Disney is what kicked off the movement https://disneyanimation.com/publications/physically-based-sh...

buildartefact 48 minutes ago

What’s hilarious is there’s nothing physically based about the Disney model. It’s empirical and It’s not even energy conserving.

As sibling pointed out, physically based rendering precedes “PBR” by a looong time and goes much, much deeper than “I put a metalness map in my shader”

nxobject 10 hours ago

Note that the book is even older than that – I remember first reading it in 2009; apparently the 1st edition was in 2004!

curiousObject 17 hours ago

But what alternative can you suggest which doesn’t break grammar or usage precedents like “physically based”?

Physics-based? Reality-based? Physically-derived?

roelschroeven 13 hours ago

Physics-based sounds perfectly fine to me.

"X-based" to me is equivalent with "based on X". Physics-based = based on physics, evidence-based = based on evidence, values-based = based on values; all perfectly fine.

Physically based feels correct in a sentence like "Our company is physically based in New York but we operate world-wide". But what does the "physically based" in "physically based rendering" mean?

But I'm not a native speaker, what do I know.

msk-lywenn 21 hours ago

It bothers me too, but I’m French. I always assumed it was some corner of the language I don’t know

taeric 1 day ago

This and https://hint.userweb.mwn.de/understanding_mp3/index.html are both amazingly fun examples of literate programming that I recommend all of the time.

WillAdams 1 day ago

I've been keeping a list of Literate Programs published as books, as well as supporting texts:

https://www.goodreads.com/review/list/21394355-william-adams...

I'd be glad to know of any I missed, or of a similar resource for websites.

granter 22 hours ago

There is of course Bob Nystrom's https://craftinginterpreters.com

WillAdams 15 hours ago

Thanks! Added it, and put a copy in my Amazon cart...

taeric 15 hours ago

Great list! Sibling addition of Nostrom's book is a great addition. Not exactly the same, but close enough that I think it makes sense on the list.