How to perform colour correction with a "ColorChecker" reference image?

[bsdis]

Hi,
I have images of a colorchecker and I would like to perform color calibration of these images.
As I understand I have to do extract the 24 colour squares of the colorchecker, Convert from sRGB to RGB by applying inverse gamma, convert RGB to XYZ using 3x3 matrix, convert XYZ to Lab using standard formula and D6500 white point - perform a regression to find conversion coefficients - and then convert all pixels using the coefficients and the finally transform the converted image back into sRGB space.
Is that correctly understood, and is that possible to do using this library? (all the color conversions - not the extractions from images etc)
If yes - are there any examples available?

thanks

[thomasmansencal]

Hi @bsdis,

It is absolutely possible and I do that quite regularly.

Assuming you have the data for the colour rendition chart, at the basic minimum, this is what you need to do:

colour/colour/characterisation/correction.py

Line 852 in c3245b1

>>> RGB = np.array([0.17224810, 0.09170660, 0.06416938])

We have a repository which does extraction and provides an example of colour correction here: https://github.com/colour-science/colour-checker-detection/blob/develop/colour_checker_detection/examples/examples_detection.ipynb

[bsdis]

Hi @KelSolaar
Ahh ok. Really nice :)
I hope its ok to ask some follow up questions:

1. In the notebook you preprocess your images with dcraw -T -g 2.4 12.92 *.CR2. This basically encodes images as sRGB png images right?
2. I ran your example on 4 test images i had of the same scene where i have changed the lighting in each scene.
   I get this result which is pretty nice actually

But it seems that especially if you look on the white background its still not the same on the "after calibration" images. Is this to be expected? Can that be fixed somehow?

[bsdis]

Here is actually a better image. I have multiplied with and capped at 255, which also shows that some channels are saturated after the calibration (First 4 are before calibration and last 4 are after).

Is this because i need to do a white balancing also?

[thomasmansencal]

Hi @bsdis,

This is a great example of why relying on a colour rendition chart for colour correction is not the solution to all the imaging calibration problems as $COLOUR_RENDITION_CHART_MAKER claims it is.

What is happening here is that your scene illumination is changing while the ColorChecker Classic values are given for a particular illuminant, i.e. CIE D50. Ideally for things to really work, your scene should be lit with CIE D50 so that you can perform colour correction against the published values.

Ideally, you would need to measure the spectral irradiance of the scene and the spectral reflectance of the ColorChecker Classic (with a spectro-photometer) and recompute its values for the particular scene.

Alternatively, something cheap is to neutralise on the bottom row after the colour correction but it is obviously not ideal!

I hope that makes sense!

[bsdis]

Hi @KelSolaar

Aha - yes that makes perfect sense. I have to keep that in mind.
If I first wanted to try the cheap method, you say "neutralise on the bottom row after the colour correction". How exactly is that done? In your code, the final image is generated by doing:

    im = colour.cctf_encoding(
        colour.colour_correction(
            COLOUR_CHECKER_IMAGES[i], swatches, REFERENCE_SWATCHES))

So I guess im is in sRGB colorspace? Would i do the neutralise directly on this image, or would I convert it to another colorspace first?
Also - would you cut a square from each of the bottom patches to neutralize after, and how would you do it?

Sorry - I hope its not to many questions.

Thanks :)

[thomasmansencal]

Yeah, so you would need to neutralise before that conversion, colour correction, white-balancing and, most of the operations must ideally be done on floating-point linear data. The idea is to compute the average RGB colour of one of the swatches of the bottom row, we commonly take the third from the bottom right in the VFX industry, divide your image by it so that this swatch is white (or close to it because you only took the average) and then multiply the image by the value of that swatch, it should be around [0.19~, 0.19~, 0.19~] for sRGB.

[bsdis]

Hi @KelSolaar

Hmm I think I might have misunderstood you.
I understood it like adding these 2 lines:

    im_ = COLOUR_CHECKER_IMAGES[i]/colour_checker.data['neutral 5 (.70 D)']
    im *= im_

So code becomes like this

for i, swatches in enumerate(SWATCHES):
    swatches_xyY = colour.XYZ_to_xyY(colour.RGB_to_XYZ(
        swatches, D65, D65, colour.RGB_COLOURSPACES['sRGB'].RGB_to_XYZ_matrix))

    colour_checker = colour.characterisation.ColourChecker(
        os.path.basename(COLOUR_CHECKER_IMAGE_PATHS[i]),
        OrderedDict(zip(REFERENCE_COLOUR_CHECKER.data.keys(), swatches_xyY)),
        D65)
    
    plot_multi_colour_checkers(
        [REFERENCE_COLOUR_CHECKER, colour_checker])
    
    swatches_f = colour.colour_correction(swatches, swatches, REFERENCE_SWATCHES)
    swatches_f_xyY = colour.XYZ_to_xyY(colour.RGB_to_XYZ(
        swatches_f, D65, D65, colour.RGB_COLOURSPACES['sRGB'].RGB_to_XYZ_matrix))
    colour_checker = colour.characterisation.ColourChecker(
        '{0} - CC'.format(os.path.basename(COLOUR_CHECKER_IMAGE_PATHS[i])),
        OrderedDict(zip(REFERENCE_COLOUR_CHECKER.data.keys(), swatches_f_xyY)),
        D65)
    
    plot_multi_colour_checkers(
        [REFERENCE_COLOUR_CHECKER, colour_checker])
    
    ##### ADDED WHITE BALANCING HERE:
    im_ = COLOUR_CHECKER_IMAGES[i]/colour_checker.data['neutral 5 (.70 D)'] # divide using broadcasting
    im *= im_
    ##########################
    im = colour.cctf_encoding(
        colour.colour_correction(
            im, swatches, REFERENCE_SWATCHES))
    b4after.append( [COLOUR_CHECKER_IMAGES[i],im] )
    plot_image(im);

But this gives me quite strange results.
What did i misunderstand here?

[thomasmansencal]

You would need to divide by the average value of the colour checker in the image, i.e. COLOUR_CHECKER_IMAGES[I] and multiply by that of the reference one colour_checker.data['neutral 5 (.70 D)']

[bsdis]

As I understand the code, the following variables are:

COLOUR_CHECKER_IMAGES[i] <--- Sample image

colour_checker.data['neutral 5 (.70 D)']) <--- average value of neutral 5 swatch of sample image

REFERENCE_COLOUR_CHECKER.data['neutral 5 (.70 D)'] <--- reference value of colorchecker for neutral 5

So: divide by the average value of the colour checker in the image, i.e. COLOUR_CHECKER_IMAGES[I] and multiply by that of the reference one
would be:

im = (COLOUR_CHECKER_IMAGES[i]/colour_checker.data['neutral 5 (.70 D)'])*\
        REFERENCE_COLOUR_CHECKER.data['neutral 5 (.70 D)']

Is that correct?

However it still gives quite different backgrounds though.

So I guess thats due to my scene illumination changing while the ColorChecker Classic values are given for a particular illuminant like you mentioned, right?

What I actually wanted to achieve was to be able to take an arbitrary image under an arbitrary illumination and and arbitrary camera, and somehow calibrate those into same colorspace, so that I could directly compare the colors in the in images. In each picture there would always be a colorchecker of course.
So I guess thats actually impossible then?

[thomasmansencal]

Is that correct?

Not really, you would need to divide each given image by its own colour checker Neutral 5 swatch average, because each image has its own illuminant, you need to neutralise for each one of them accordingly and individually.

So I guess thats due to my scene illumination changing while the ColorChecker Classic values are given for a particular illuminant like you mentioned, right?

Assuming you have neutralised each one of them, you will see that not only their whites are closer, but also that their exposure is roughly the same! You will still have discrepancies though because of the illuminant differences.

[bsdis]

Hi @KelSolaar
Hope its okay i keep this thread running.
In your notebook (https://github.com/colour-science/colour-checker-detection/blob/develop/colour_checker_detection/examples/examples_detection.ipynb)
you do:

    plot_image(colour.cctf_encoding(
        colour.colour_correction(
            COLOUR_CHECKER_IMAGES[i], swatches, REFERENCE_SWATCHES)));

the output of colour.cctf_encoding seem to be in the range -0.52 to 1.47. Should that not be in the range 0 to 1?