Hacker Remix

Issues with color spaces and perceptual brightness

131 points by todsacerdoti 2 months ago | 64 comments

jorvi 2 months ago

I've always found these "perceptual vs absolute" things about human senses very interesting.

Hearing has a few quirks too:

- When we measure sound pressure, we measure it in log (so, every 3dB is a doubling in sound pressure), but our hearing perceives this as a linear scale. If you make a linear volume slide, the upper part will seem as if it barely does anything.

- The lower the volume, the less perceivable upper and lower ranges are compared to the midrange. This is what "loudness" intends to fix, although poor implementations have made many people assume it is a V-curve button. A proper loudness implementation will lessen its impact as volume increases, completely petering off somewhere around 33% of maximum volume.

- For the most "natural" perceived sound, you don't try to get as flat a frequency response as possible but instead aim for a Harman curve.

- Bass frequencies (<110Hz, depending on who you ask) are omnidirectional, which means we cannot accurately perceive which direction the sound is coming from. Subwoofers exploit this fact, making it seem as if deep rich bass is coming from your puny soundbar and not the sub hidden behind the couch :).

mrob 2 months ago

A 3dB increase is a doubling of a power quantity, but you need 6dB to double a root-power quantity (formerly called a field quantity). Sound pressure is root-power quantity:

https://en.wikipedia.org/wiki/Power,_root-power,_and_field_q...

Consider a loudspeaker playing a constant-frequency sine wave. Sound pressure is proportional to the excursion of the cone. To increase sound pressure, the excursion has to increase, and because frequency is fixed the cone will have to move faster. If it's covering twice the distance in the same time interval it has to move twice as fast. Kinetic energy is proportional to the square of velocity, so doubling the sound pressure requires four times the power, and doubling the power only gets you sqrt(2) times the sound pressure.

Human loudness perception generally requires greater than 6dB increase to sound twice as loud. This depends on both frequency and absolute level as you mentioned, with about 10dB increase needed to double perceived loudness at 1kHz and moderate level.

lieks 2 months ago

I have a theory that this generalizes to some extent.

Pitch perception is also logarithmic; an octave in music is a 2x ratio.

Memory is sort of logarithmic; you'd say a thing happened 1-2 days ago or 1-2 years ago, but not 340-341 days ago.

Same with age; someone being 10 years older than you is a much bigger deal when you're 10 than when you're 80.

01HNNWZ0MV43FF 2 months ago

It does! https://en.wikipedia.org/wiki/Stevens's_power_law

> Stevens' power law is an empirical relationship in psychophysics between an increased intensity or strength in a physical stimulus and the perceived magnitude increase in the sensation created by the stimulus

andrewla 2 months ago

I've played around with this a little bit while trying to engineer the "perceptually loudest ringtone sound". I thought a square wave or pulsed white noise would work well, but I found that the bell-like "telephone ring" seemed perceptually louder.

I now have to read up on the Harman curve because I'm curious about this and it has a direct practical application.

ttoinou 2 months ago

    When we measure sound pressure, we measure it in log (so, every 3dB is a doubling in sound pressure), but our hearing perceives this as a linear scale

Its the other way around : we perceive logarithmically so we created the logarithmic decibel scale

Animats 2 months ago

This is part of "tone mapping"[1] in high dynamic range rendering. The idea is that pixels are computed with a much larger range of values than screens can display. 16 bits per color per pixel, or even a floating point value. Then, to generate the displayed image, there's a final step where the pixel values are run through a perceptual transformation to map them into 8 bit RGB (or more, if the hardware is is available.)

This has issues. When you go from a dark space to a bright space, the eye's iris stops down. But not instantaneously. It takes a second or two. This can be simulated. Cyberpunk 2077 does this. Go from a dark place in the game to bright sunlight and, for a moment, the screen becomes blinding, then adjusts.

In the other direction, go into a dark space, and it's dark at first, then seems to lighten up after a while. Dark adaptation is slower then light adaptation.

Tone mapping is not just an intensity adjustment. It has to compensate for the color space intensity problems the OP mentions. Human eyes are not equally sensitive to the primary colors.

Some visually impaired people hate this kind of adjustment, it turns out.

Here's a clip from Cyberpunk 2077.[2] Watch what happens to screen brightness as the car goes into the tunnel and then emerges into daylight.

[1] https://en.wikipedia.org/wiki/Tone_mapping

[2] https://youtu.be/aWlX793ACUY?t=145

TeMPOraL 2 months ago

> Here's a clip from Cyberpunk 2077.[2]

This looks nothing like actual Cyberpunk 2077. It looks actually realistic - unlike the base game itself where, just like in about every other game since "Physically Based Rendering" became a thing, everything looks like it's made of painted plastic.

Per the video description, this version of C2077 is modded up to its eyeballs. I'm very interested in those mods now, because it's the first time I ever saw someone managed to get materials in a game look somewhat realistic.

Animats 2 months ago

I'm amazed how much was accomplished by "modding".

Auto paint looks great. They probably have a clearcoat layer. Skin still looks bad, though. I suspect that Cyberpunk 2077 lacks a subsurface scattering layer in the content. Is that correct? If there's no info there, rendering mods can't improve on it. It's worth thinking about using some Midjourney type system to generate subsurface scattering layers for existing models and make skin look better.

I've used something called "Deep Bump", which uses machine learning to generate normals for flat images. It's great for brick and stone, OK for bark, poor for plants. That probably reflects the training set, and could be improved.

myfonj 2 months ago

Could this be the reason I immensely dislike when webpages include blocks of inverse background/text, or generally combine light-text-on-dark with dark-text-on-light elements?

tshaddox 2 months ago

> Cyberpunk 2077 does this. Go from a dark place in the game to bright sunlight and, for a moment, the screen becomes blinding, then adjusts.

Or, very famously 20 years ago in Half-Life 2: Lost Coast, which was released as a tech demo for HDR rendering.

fredgrott 2 months ago

this is why Google's HCT color space has both a Tone as adjustment and device environment as a color adjust.

refulgentis 2 months ago

hey, I made that! I wish the environment adjust was used, but it's not. Tone in HCT is ""just"" the L* criticized in the article (opinionated comment on that here, TL;DR: talking about color sucks, lightness/brightness are loaded terms, and the article is off-base due to it https://news.ycombinator.com/item?id=42738943)

kookamamie 2 months ago

See Oklab colorspace for an attempt at fairer perceptual brightness: https://bottosson.github.io/posts/oklab/

kpil 2 months ago

This page was interresting and demonstrates some problems with CIELabls hue perception: https://raphlinus.github.io/color/2021/01/18/oklab-critique....

refulgentis 2 months ago

A significant mis-step - naively eschewed actual lightness, the thing that dictates what people actually perceive, because people outside the field regularly conflate brightness and lightness.

andrewla 2 months ago

Oklab is mentioned in the article as also being flawed in this sense.

Conscat 2 months ago

I was looking out for a mention of Oklab and didn't see it while reading. If anyone else is wondering, you have to click on the [1] footnote to see it.

weinzierl 2 months ago

"Unfortunately, I haven’t been able to find any perceptually uniform color spaces that seem to include these transformations in the final output space. If you’re aware of one, I would love to know."

OSA-UCS takes the Helmholtz-Kohlrausch effect into consideration.

cccbbbaaa 2 months ago

That, or the more recent darktable UCS: https://eng.aurelienpierre.com/2022/02/color-saturation-cont...

mattdesl 2 months ago

Although the article is intimidating, the implementation appears pretty simple and flexible. Has anyone tried using this either in the Darktable app or in their own libraries? I might like to try porting it to JS to compare it for web based color picker GUIs.

cccbbbaaa 2 months ago

This colorspace is used internally by the color balance RGB module of Darktable (and Ansel).

Kleptine 2 months ago

This is an incredible reference, thanks.

balou23 2 months ago

Someone else was mentioning Oklab color space, and I was wondering what the difference was to darktable UCS.

TL;DR: Oklab is pretty simple, but is already pretty nice as a perceptually uniform color space. Darktable UCS takes Oklab and tries to reduce the residual error.

Feel free to correct me if I got anything wrong

weinzierl 2 months ago

Not sure. The article is long and I only skimmed it, but what stood out to me were the following paragraphs:

"After trying to fix Oklab for a dozen of hours, it appeared that the numerical issues it raises are grounded into design constraints we don’t need for the current task. And so do most of the other perceptual spaces.

[..]

So we could fit an Lch model directly from Munsell hue-value-chroma dataset, without going through LMS space and without even tilting the lightness plane. Doing so, we will not try to model the physiology of vision, but approach the problem as a geometric space distortion where the Munsell hues and the saturations (as a ratio of chromas / values) can be predicted by a 1D mapping from the hues, chromas and lightnesses of forming the principal dimensions of the model.

This model will need to be invertible. We will try to fit brightness data to derivate correlates of perceptual brightness accounting for the Helmholtz-Kohlrausch effect. Using the brightness and saturation correlates, we will rewrite our image saturation algorithm in terms of perceptually-even operators."

They obviously took a lot from Oklab but it seems to me they did more than just modifying it to reduce the residual error. But again, I just skimmed and I can be completly wrong.