** Warning: No correlation between the master dark and target frames (channel 0)

[cgeib]

I'm trying to calibrate a series of images I took last night. I created 10 darks after the imaging session and integrated them per the PI tutorial on Master file creation. But for most of the images during calibration I'm seeing this warning and I've not seen this before. What is it telling me? What do I need to fix if anything? Thanks in advance.

Chris Geib

[bitli]

To try to figure out the root cause of the problem I would need to have more details on your calibration process: which sequence of steps did you do for calibration/integration of bias/dark.  There can be an error or issue in multiple places.
There are also a few threads on this subject, look if they can help.
And finally the (hopefully) helpful section, assuming that you calibrate the lights with some master flat/bias/dark.
First the context:

• In a classical (non optimized) calibration, your dark must have the same exposure and temperature than your lights and they are supposed to represent faithfully the additive 'dark current', which is more or less the same on all images. The calibration just subtract the master dark. You do not need to calibrate the darks for this operation.
• Other astro programs than PI generally uses the term 'optimized' calibration to use a dark made for a different exposure time (preferably longer) and the program scales it. The program uses the ratio of the effective and desired exposure time as a factor. This assumes that the dark current is proportional to the time of exposure. For this to work the bias must first be subtracted from the dark, as the scaling factor assume that a zero exposure result in zero values.
• In PI the optimization is more subtle. PI does not use the time or temperature to find the scaling factor.  Instead it tries a bunch of factors and use the best one (I somewhat simplifies). How does it find the 'best' one? By minimizing an estimate of the noise in the resulting image.  The details are explained somewhere, but I am not even sure I understand them fully. Anyhow it will print the factor on the console and you should always check if it is in an expected range (for example if you darks are around the same time and temperature of your images, the factor should be near 1 (say 0.8 to 1.2, but not 0.05 !). This methods requires a very good bias (low noise AND correct, this second aspect being often forgotten). It is recommended that you take at least twice the number of bias than dark. It assumes that the dark current is linearly proportional to some factor that it tries to discover. If it find that the best case is when not subtracting the dark (a scale factor 0 or very very small), then it do write the Warning: No correlation between the master dark and target frames. For all practical purpose you could as well not have bothered with darks for that image from PI perspective.

Now the possible problems. Beyond gross errors (taking the wrong image set or, like I did, taking the dark at home with a program that saved them in the other direction than the lights taken at the telescope) and operational errors (subtracting the bias twice from the darks and so on) the common issues are to be looked in the assumptions of the different methods.

• Do your original bias and dark faithfully represent a zero current and dark current? Look at the median of these images. On DLSR (and some CCD) you will see that median of the bias is the same or even higher than the median of dark.  Subtracting the bias will truncate many values and make it not correlated (not proportional) to your lights. In that case you are better of not using optimize or you can play with PEDESTAL and the like.
• How does your master dark look? Does it have a lot of values at zero ? You may have the problem above, a double calibration or the noise of your dark make their low pixels lower than the bias (with very short darks - you probably do not need dark).
• Blink your dark and images.  The hot pixels should largely be at the same place. Otherwise you are not working with the dark you expect.
• If it happens on a single image/channel, check that the image itself is correct

The PixInsight 'optimize' has some limitations. Because the assumption of linearity is not perfect, hot pixels are not so well corrected.  However they are easy to remove at integration time (with dithering) or with CosmeticCorrection.
All rescaling will work better scaling down (using a long dark for a short exposure) than up (interpolation is more reliable than extrapolation).
It also optimize some noise characteristic, which is probably the best criteria for many good CCD.  On a DSLR you may want to optimize the rejection of amplifier noise. 
You can get rid of the message by disabling 'optimize', but first make sure that you understand if your dark is good or bad, or if the optimization does not help in your case.
-- bitli

[cgeib]

Thanks for all the information. I took these darks at the end of the imaging session at the same exposure and time as the lights. I manually integrated them using the settings found here: http://www.pixinsight.com/tutorials/master-frames/index.html. I don't use the batch processing script so I manually calibrated the lights. I did use the Master Bias I have and the master flat I created earlier in the day. So, I might go back and try calibrating without the Bias based on your comments and see if that resolves it. I did not have "calibrate" checked but I did have "optimize" checked. I will try with it unchecked as well. I will have to go and do some research for the other comments you mentioned.

Thanks again for your help.

[chris.bailey]

I think you need to check the calibrate tick box in the dark section of Image Calibration so that the Bias is subtracted from the Dark leaving only the Dark signal to be subtracted. If you don't tick it and you have not already Bias subtracted your darks, you are effectively subtracting the Bias twice, once on its own and once as a component of the Dark. You do need to subtract a master Bias from the Master Dark before optimisation can work otherwise it is trying to scale the Dark and the Bias.

[Alejandro Tombolini]

Hi bitly, excellent explanation. Thanks for such so clear information!
Saludos, Alejandro.

[Skywatcher76]

Hi everybody, I keep getting the same message with no correlation between darks and target. I've tried both batch preprocessing and manual calibration and integration tools. Fault appears only if I try to optimize dark. It goes smoothly if I uncheck darks optimization. Any thoughts?

[vicent_peris]

Hi,

Try to increase the optimization threshold to maybe 4-5. Anyway, it would be good to review your master dark and one of your lights. Usually this kind of problem tells you that there was a problem in the acquisition and therefore in your master calibration images. We could better diagnose by taking a look at the images.


V.

[jkmorse]

One thing to check is to make sure that your lights and your darks are set to the same binning mode.  1x1 binned darks will not work with 2x2 binned lights.

[Skywatcher76]

I never try to calibrate differently binned frames. Here is download link to master dark file https://drive.google.com/open?id=0B_OydTfcPUdoTWRTdUN0NFVMM2M,  and  one of the light frames https://drive.google.com/open?id=0B_OydTfcPUdoUmRQdUxtSEhzYzQ, any help is appreciated. I have a library of darks, try to renew them once every two months. My camera is QSI683wsg8. Thank you very much.

[vicent_peris]

Hi,

The dark is OK, but I would say it still has the bias. If you want to scale the dark subtraction from the light frame, the dark should be always bias subtracted. I guess you're running manually the ImageCalibration process. In that case, load both the master dark and the master bias and activate the "Calibrate" check box in the master dark section.


Hope this helps,
Vicent.

[Skywatcher76]

Thank you Vincent. I will try and will let you know about result

[EricC]

I was also getting the "no correlation" message while creating my master flats, following the instructions at https://pixinsight.com/tutorials/master-frames/index.html.  My flats were from 3 to 20 seconds depending on the filter; they were all taken at -20 C.  The master dark I used to calibrate the flats was 900 seconds at -20 C.  Most of the "k0" scaling values were 0.03 or something very small; the ones that were 0.08 did not produce the warning.

My fix was to calibrate with a master dark of 20 seconds at -20 C.  My "k0" scaling values were close to 1.0.
I believe the problem with using the 900 second dark to calibrate the flats was that it was from 18 to 300 times as long as the flats, so it makes the k0 number too small to be usable.