Why doesn't color calibration balance the colors? Adjusting ASI294-images

magnusl

Well-known member
Oct 10, 2015
97
0
Hi!

I am a new owner of an ASI 294 OSC camera, and with this camera came the interesting challenges of adjusting colors after acquisition. As a detail, I have set the channels all to 0.5 in the driver (I use Ekos/Indi).

After calibration and integration, I typically end up with a blue or yellow image - the R, G and B channels are not properly balanced. Partly caused by the camera, and partly by my flats perhaps not having the preferred white balance (different ways to produce flats clearly affect the resulting color).

I have now tried several ways to balance the colors, and it seems to work quite nicely. I can:
- split into separate channels, do a LinearFit, and recombine them
- use DBE, that effectively removes the color
- use background neutralization, same result
- manually adjust the colors with Histogram Transformation

However, one tool that consistently fails to adjust the color is ColorCalibration. I use a small preview without stars as a background reference and set the whole image as white reference. Then use default settings. But the color remains. See attached image - color not perceptibly affected by the process, only made darker.

CC.png

Any suggestions warmly welcome for:
- why is not CC adjusting the color? What am I missing?
- which process would be the best for doing this with this camera?


Magnus
 

pfile

PTeam Member
Nov 23, 2009
5,311
64
check that the background upper limit is high enough to capture the brightest channel of the background.

rob
 

magnusl

Well-known member
Oct 10, 2015
97
0
Hi!

Thanks for your suggestion. Using a small preview, the statistics window tells me the max value there is 1.9 e-02, which I take to be less then 0.1. However, increasing the uppler limit to 0.5 does not improve the result either. Hrm. Maybe something else?

I've uploaded the integrated image for anyone to play with:


Apart from this problem with ColorCalibration: Any ideas on which process to adjust colors that is better?

Magnus
 

bulrichl

Well-known member
Nov 2, 2016
748
45
La Palma, Canary Islands
OK, step by step.

1) Please at first disable the option 'Link RGB channels' in ScreenTransferFunction and apply an Auto STF. You will notice a strong brightness gradient from left to right. Therefore apply AutomaticBackgroundExtraction (ABE) with Function degree 1 and subtractive correction.

2) Now a border area of about 180 pixels in width with high noise becomes visible. This area shall be cropped (Crop process, left -180).

3) For color calibration use PhotometricColorCalibration (PCC):
Bring the original integration to the foreground (it shall be the current view) and in PCC, section 'Image Parameters', press the button 'Acquire from Image'. Bring the ABEd and cropped integration to the foreground again. Make a preview for background neutralization (I took left 3107, top 1362, width 38, height 26): Preview01. In PCC, section BackgroundNeutralization, enable 'Region of Interest', press 'From Preview' and select this Preview01. Then apply PCC to the DBEd and cropped integration.

Don't get frightened by the look of the result. The image is displayed with the Auto STF before color calibration. Now you have to enable the option 'Link color channels' again and apply an Auto STF.

The result has some ugly dark areas, I don't know where that comes from. If these artifacts cannot be removed, the color calibrated image has to be cropped, e.g. left -800, top -340, right -640, bottom 0.

The result looks not bad.

Bernd
M27.JPG
 
  • Like
Reactions: pfile

dld

Well-known member
Sep 24, 2017
161
14
Hello Magnus,

I followed a similar workflow as Bernd but used ColorCalibratrion instead. I used the whole (well, cropped, to avoid the band at left and black spot) image as Background Reference, reaching a very similar, if not the same result. Despite the dark areas Bernd mentions, I think there are also some remnants surrounding M27, probably from your not-so-good flats. If you use a small preview as Background Reference, you will obtain a slightly different result depending on the location of your Background Reference preview.
 

magnusl

Well-known member
Oct 10, 2015
97
0
Hi!

OK, thanks, so both of you prefer to use a background extraction (Bernd suggest ABE) and from there on the "tint" is effectively removed I guess (at least it is when I do it) as well as the light pollution gradient. Great, thanks!

Still though, I am puzzled as to why the ColorCalibration does not remove it and adjust the colors, when used as first process. Should it not? I believe it did that on some of my first test images at least. Or am I confusing things?

As for flats: yes, there are some stuff here that are not corrected the way they should be. Flats with this camera seems to be challenging, since I need to take rather long exposures (3-5 secs) - and this means that I need to find a good way to dim my EL-panel and at the same time make sure the light is, well, flat... This is with a C8 så rather big opening, and perhaps my A4 EL-panel is not big enough. Experimentation continues. Step by step I'm learning to use this camera with my scopes. This color challenge is one important step, flats is another.

Magnus
 

dld

Well-known member
Sep 24, 2017
161
14
Still though, I am puzzled as to why the ColorCalibration does not remove it and adjust the colors, when used as first process. Should it not? I believe it did that on some of my first test images at least. Or am I confusing things?
Because if the background varies due to light pollution, no Background Reference sample would be representative for the entire image. Think what happens these days when we image the C\2020 F3 NEOWISE comet with wide-angle lenses: The part of the image which is closer at the horizon will be redder due to atmospheric extinction. If we sample that area, the rest of the image will become bluer after ColorCalibration.
 
  • Like
Reactions: pfile

magnusl

Well-known member
Oct 10, 2015
97
0
Hi!

No, I am sorry. That can not be the case. I choose previews from all over the image, and consistently get the same yellowish tint on the image, although somewhat darker. Do you get something else, playing with my image? My feeling is that there is something else going on here, or I am missing something obvious. If you really can change the color with ColorCalibration, then I must do something wrong.

THat is why I posted the image - do you get the same results as I with ColorCalibration (same strong yellow tint but darker), or can you really remove or change the color balance on the image with this process? Yes, background extraction (as a number of other processes) does it - but why not Color Calibration?

Magnus
 

bulrichl

Well-known member
Nov 2, 2016
748
45
La Palma, Canary Islands
I really don't understand what you want. The ABE is necessary because of the strong brightness gradient in your image, and I stated the reason for it before.

Of course you can apply PCC as the first process. However, the color calibration will be less correct in this case. If you apply the workflow omitting the ABE step (i.e.: Crop, PCC, Crop) you'll get the following result (Auto STF with 'Link RGB channels' enabled):

M27_02.JPG

There is NO yellow background, just a strong gradient.

Note that in this case the STF parameters are much less aggressive than in the image in post #4. If one tries to adjust the STF parameters for similar appearance of the planetary nebula as in the image in post #4, the result will look like like this:

M27_03.JPG

and I don't believe that THIS is what you wanted.

So I guess you didn't reproduce at all what I recommended in post #4 of this thread.

Bernd
 

magnusl

Well-known member
Oct 10, 2015
97
0
Hi!

Yes, thanks. What I want is quite simple (actually two things but the second has been answered): to understand why I cannot remove the yellow background in this image with ColorCalibration. Yes, I did replicate your process, and it works nicely. And as I stated in my first post, I can easily remove the yellow background and the gradient through several different processes. And yes, my PCC also removes the yellow background in my image.

But my question here does not concern how to process this image specifically. I am trying to understand what is happening to ColorCalibration here - I want to understand the processes involved sufficiently to be able to choose wisely. I am using this image as an example, since here, when I apply ColorCalibration, I do not get the result I expect. That is what puzzles me.

More generally (my second question) concerns if there are important differences between adjusting color channels through:
1. the color calibration processes (CC or PCC)
2. background modelleing tools (ABE or DBE)
3. and other processes such as HistogramTransformation or the LinearFit on split R, G and B channels.

Again, I observe that several seems to work. I am interested in if there are less obvious consequences of going one way rather than the other (because of different mathematic and principles).

Here I understand that you prefer number 2 as first step, and that is very helpful! Thanks for that.

Magnus
 

bulrichl

Well-known member
Nov 2, 2016
748
45
La Palma, Canary Islands
Why didn't you say so? ;)

These problems are due to the application of the ColorCalibration (CC) process to an image that had a not neutral mean background because neither BackgroundNeutralization (BN) nor one of the BackgroundModelization processes (ABE or DBE) were applied. The application of at least one of these processes before the ColorCalibration (CC) process is essential. In his book Warren Keller writes:

"For it [ColorCalibration] to work, three conditions must be met:
1. The image must have even illumination - this was achieved via flat-fielding and BackgroundModelization.
2. The mean background must be neutral - that was accomplished by BackgroundNeutralization.
3. The image must be linear - nonlinear stretches have yet to be applied."

As I stated before, in the specific case of your image (severe gradient) it is necessary to apply ABE (Function degree 1, subtractive correction) before BN.

Generally, if you want to apply CC the workflow will be:
1. ABE or DBE for flattening the Background,
2. BN for neutralization of the background,
3. CC for color calibration.

Step 1 can be omitted when there is no gradient in the image, step 2 must not be omitted.

BN performs an additive correction, and CC performs a multiplicative correction.

I hope this helps.

Bernd

P.S.: The precondition of a neutral mean background is also stated in the PixInsight Reference Documention for the ColorCalibration process. It can be displayed either by clicking on the Browse Documentation icon (the sheet of paper icon at the bottom right of the process window) or by calling the Process Explorer and choosing the appropriate process from the select list.
 
Last edited:

magnusl

Well-known member
Oct 10, 2015
97
0
Hi!

Yes, well, that's what I did ask in my first post.... ;)

Yes, that is clear and good!! Actually, I also just found a very good discussion of this in Andreo's Mastering PixInsight-book, where he discuss different possible workflows for color calibration. Can be highly recommended for anyone thinking about these issues.

And I do get the logic of dealing with gradients first. So my overall question is thus clarified!!

It leaves the smaller and maybe moot question of why the yellow background is not (much) affected by CC. I do understand that CC gives a bad result due to gradients, but I am surprised by the consistently yellow in this case. I am not convinced this can be produced only by the gradient - in that case, the color should shift, depending on where I place my background preview - but what I get is a consistent yellow image. Anyway, we can leave that to rest in my repertoire of unsolved mysteries of life... :) CC works nicely (as do PCC) after a backgroundextraction process.

Magnus
 

bulrichl

Well-known member
Nov 2, 2016
748
45
La Palma, Canary Islands
Hi Magnus,
It leaves the smaller and maybe moot question of why the yellow background is not (much) affected by CC. I do understand that CC gives a bad result due to gradients, but I am surprised by the consistently yellow in this case. I am not convinced this can be produced only by the gradient - in that case, the color should shift, depending on where I place my background preview - but what I get is a consistent yellow image. Anyway, we can leave that to rest in my repertoire of unsolved mysteries of life... :) CC works nicely (as do PCC) after a backgroundextraction process.
by now I believe that you are right. I took your integration after cropping (left -180) as the starting point. Then I applied CC (default parameters except for Background Reference/Upper limit 0.0220). As you also noted, the resulting image has a strong orange color. CC ouputs the calculated parameters to the process console. The calculated parameters are all correct: the background references B_X are the subtractive correction factors and the white balance factors W_X are the multiplicative correction factors of the corresponding channel (X = R, G and B). So CC should be able to produce a well color calibrated image with neutral mean background in one step, without the need of preliminarily exectuing BN.

I made up a PixelMath equation which subtracts B_X from each channel, multiplies with W_X, then adds the minimum of the minimum intensity values of the channels (in this case: the minimum of the blue channel) and rescales the result. Applying this PixelMath operation to your cropped integration resulted in a neutral background (the gradient is of course still present). In fact it produces the same result as applying BN first and CC subsequently. So apparently there is a bug in the ColorCalibration process. I will create a bug report for the ColorCalibration process.

Bernd
 
Last edited:

dld

Well-known member
Sep 24, 2017
161
14
Magnus, sorry for not understanding the intent of your post.

From the ColorCalibration documentation:
The mean background should be neutral. This is particularly important for the white reference image. Actually, a neutral background reference is not a necessary precondition for the ColorCalibration tool to work correctly; for example, in theory background neutralization could be applied after color calibration. In practice however, a non-neutral background will always contaminate the white reference to some extent, since due to the limited signal-to-noise ratio there is always some uncertainty in the selection of white reference pixels. With a neutral background, even if some background pixels enter the set of white reference pixels, their contribution in terms of color balance will be statistically insignificant, and hence the white reference will be more robust. To neutralize the background, the tool of choice is BackgroundNeutralization in PixInsight.
So what we are seeing here is contamination of the white reference pixels?
 

bulrichl

Well-known member
Nov 2, 2016
748
45
La Palma, Canary Islands
Hi Magnus,

you will have seen it in the other thread ( https://pixinsight.com/forum/index.php?threads/colorcalibration-not-working-as-expected.14957/ ): Juan confirmed that CC is working as expected. The preconditions for the application of CC are given in the corresponding PixInsight Reference Documentation, section 1.2: the image must be linear, it must be flat and the mean background has to be neutral. This is achieved by an appropriate image calibration, if necessary application of BackgroundModelization (DynamicBackgroundExtraction or AutomaticBackgroundExtractor) and the application of BackgroundNeutralization. Cited from the other thread: "BackgroundNeutralization should always be used before ColorCalibration. "

Bernd