The explanation in the link doesn't make sense to me, please give an example for discrete values (e.g. an 8 bit range)

Maybe this post by Alex from ML can shed some light: https://groups.google.com/forum/#!topic/ml-devel/3qTOXhj5FA8

I'd put this into the serious consideration category - the dynamic range of the chip is already somewhat restricted - so any in camera processing that can increase dynamic range at the top end or reduce noise in the shadow areas would definitely be welcome.

@Bertl: The idea is that the data, being captured in a scene-referred / sensor linear manner, becomes occluded near the top end. I am not entirely sure this applies to the current design, but I will outline the general concept.

We arbitrarily assign a full photosite a value of 1.0. Given a sample fictional debayer resulting in RGB values of .95, 1.0, and .84, the output has a problem. When a channel is full bin, in terms of scene linear values, we cannot accurately guess what the ratio of, in this case, the green was to the other two channels. Typically, consumer grade imaging products will clip the entire output debayer to display referred achromatic value of maximum white.

Of course, if we go in and look at the data, we know that there is valid data in the RGB triplet, but only on the blue and red channel. Sometimes only a single channel may have useful data. The issue is that if we "recover" this data and attempt to use it "as is", we end up with an RGB value that is tainted with an incorrect resultant color due to the pinned values being incorrect in the ratio.

The general "highlight recover" is to either pull all the channels down to the remaining data channel, or average or some other alchemy.

Of course both of these options are sub-optimal as they are actually delivering incorrect data in terms of scene referred data. In the display referred domain, they might be marginally useful to roll off a highlight. This still leaves problems in trying to reposition the pinned values back to their respective scene referred values for post production imaging.

As I understand it, scope for this is debayered data. Raw (post) processing could use any other (more clever) strategy to enhance highlights, something we don't want / can't afford to do in-camera.

If (and only if) we find this useful for the non raw use case, why not.

Highlight recovery is only of use when recording debayered (non-raw) images since highlight recovery can be carried out in post on raw data. However the process itself needs to be carried out on the Raw data stream itself.

Still doesn't make any sense to me, as there never is any missing data for any channel.

If the main goal is to avoid coloured highlight (i.e. one channel being saturated and thus tinting the bright areas) then there is a simple way which was already suggested several times (only applicable for adjusted live view):

• Select a safe range for all channels (e.g. 0-240 of 255)
• When any channel goes over the limit (240), increase the other channels proportionally till they all meet at 255

Note that this will make any saturated colour white, which might not be the desired outcome, but any guessing has some corner cases.

Best,
Herbert

You're not quite getting it. It would probably easier for you to understand by looking at an example of it in action. If you had a raw data sample in DNG format of a clipped source (such as a domestic light near a white wall) you could load it into Resolve and try turning hightlight recovery on and off - then you'll see the detail appear and disappear in pixels near to the point of clipping (it's often more dramatic with off white lights)

Two things worth remembering here: we're talking about the recovery of detail i.e. Mainly Luma information (although some chroma information can also be recovered).

The point being that, usually, when one channel clips, the 'combined' pixel (i.e. The debayered hypothetical pixel) is considered to have hit peak white - and would be indistinguishable from a white pixel that has all 3 channels clipping - because there is no way of knowing how much brighter this clipped pixel is relative to other clipped pixels.

Whereas in reality, this partially clipped pixel should appear to be slightly darker than this other pixel that is clipped on all 3 channels - i.e. slightly grey. The point of Highlight recovery is to use the data from unclipped channels to guess the 'correct' value of the clipped channel to give a more accurate representation of partially clipped pixels.

In terms of saturation - as a general rule you actually want to be rolling off the saturation as luma increases anyway as this is perceived as more 'natural' (this is characteristic of the much loves 'Arri' look. Increasing saturation to the point of clipping is more characteristic of the much maligned 'Sony' look).

So, let's look into two example cases:

1. A red baloon which hits 100% on red (255/10/10)
2. A yellow light which hits 100% on red (255/254/10)

What can we guess from the remaining channels (10/10 and 254/10) for the red channel?
What happens with the guessing when the yellow light hits 100% on green as well?

In T244#11419, @Bertl wrote:

Still doesn't make any sense to me, as there never is any missing data for any channel.

If the main goal is to avoid coloured highlight (i.e. one channel being saturated and thus tinting the bright areas) then there is a simple way which was already suggested several times (only applicable for adjusted live view):

• Select a safe range for all channels (e.g. 0-240 of 255)
• When any channel goes over the limit (240), increase the other channels proportionally till they all meet at 255
  
  Note that this will make any saturated colour white, which might not be the desired outcome, but any guessing has some corner cases.

I think as we settled to not do whitebalacing at all when recording raw data this task is obsolete.

In camera whitebalacing is possible through the 4x4 matrix but will result in data loss and so should only be utilized for previewing/live view purposes.

As I understand it, the approach generally relies upon approximations from adjacent, unclipped pixels.

So, maybe, in the example you provide above, you might have an adjacent pixel that is 253, 249, 5 - so a reasonable approximation would be: 258, 248, 10 - just from adding in the average of the increase from the other two unclipped channels.

But I'm really not an expert on the process. I'm sure there'll be academic papers that go into more specific detail.

Just to follow up from Sebastian - yeah, it wouldn't be necessary to do it to Raw data - as it is better handled in post.

But I'm assuming in many cases people will be recording a log encoded HDMI stream, rather than raw data - in which case this would help

But I'm assuming in many cases people will be recording a log encoded HDMI stream, rather than raw data - in which case this would help

Even then we would recommend to keep whitebalance off for the signal to the external recorder. Either you apply a LUT to simulate the image having certain color temperature or use a second HDMI output on the Beta for live preview and the whitebalance turned on just for this live preview HDMI signal.

Some form of highlight recovery would still be useful in this instance though - albeit not one based on white balance.