Author Topic: New PixInsight Tutorial: Pre-processing (Calibrating & Stacking) your Images  (Read 16997 times)

Offline kayronjm

  • PixInsight Addict
  • ***
  • Posts: 100
    • Light Vortex Astronomy
Read here: http://www.lightvortexastronomy.com/tutorial-pre-processing-calibrating-and-stacking-images-in-pixinsight.html

I have finally been able to finish my pre-processing tutorial for PixInsight. This tutorial covers EVERYTHING from start to finish of how to take your raw exposures (bias, darks, flats and lights), produce masters and use them to calibrate your lights, followed by stacking them including using the Drizzle algorithm to remove pixelation in the final result. The fully manual method is covered in detail. The tutorial also covers use of the BatchPreprocessing script for anyone interested in it. Additionally, there is a discussion of what to do with multiple sets of data for the same object captured over various nights. Enjoy! :)

[EDIT]: Screenshots have been updated to display at their full-resolution as captured. It should be much easier to follow this tutorial and my other current ones now.

[UPDATE]: Practically re-wrote sections 4 and 5 in order to re-arrange how the PixInsight pre-processing workflow is applied. Calibrating light frames is now recommended prior to cosmetic correction. Added note about how master dark subtraction from flat frames during the calibration of flat frames can sometimes result in a PixInsight warning about no correlation. Made amendments throughout the text and on various screenshots to properly reflect the changes made.
« Last Edit: 2015 December 27 08:35:44 by kayronjm »
- Avalon M-Uno
- Takahashi FSQ-85ED, Altair Astro 8" RC with Astro-Physics CCDT67 Telecompressor
- QSI 660wsg-8, Starlight Xpress Lodestar X2
- Astrodon E-Series Gen2 LRGB 1.25", Astrodon HA, OIII & SII 3nm 1.25"

Offline pfile

  • PTeam Member
  • PixInsight Jedi Grand Master
  • ********
  • Posts: 4729
just one comment - the master bias does not have too much to do with read noise. the bias signal is related to the very first step in preparing a CCD to generate photoelectrons - a positive bias voltage is applied to each photodiode. this way, when a photoelectron is liberated by an incoming photon, the electron goes somewhere and can later be counted. without the bias voltage eventually the photoelectrons would just randomly recombine with the holes that were left behind.

the bias signal is essentially a fixed pattern that is a characteristic of the CCD. the true fixed pattern of the bias signal is what the master bias is intended to capture - since the read noise is essentially random noise, you are driving the SNR (bias signal : read noise) up by stacking bias frames in the same way you drive the SNR of your DSO photons (DSO signal : DSO shot noise) when you stack lights. i think there is also some random variation in the bias signal itself, so by stacking you are also homing in on the true value of the bias signal.

in fact, if you subtract a master bias from an individual bias frame, the result is an image of the read noise in that single bias frame. essentially the read noise is always present in every frame generated by the camera, and can not be removed. that's the reason why exposure lengths are chosen such that the exposure is 'sky limited' rather than read-noise limited. you're trying to get the actual signal you're capturing up above the read noise threshold of the camera so that the DSO signal is not destroyed as the chip is read out. by stacking the bias and dark subs you're just trying to avoid injecting more noise than necessary when calibrating the lights.

rob

edit: clarification of the nature of the bias signal
« Last Edit: 2015 July 14 20:55:06 by pfile »

Offline dnault42

  • PixInsight Addict
  • ***
  • Posts: 125
Very thorough tutorial.  I like that you covered BPP as well to show the connection with manual processing.

Regards,
David

Offline kayronjm

  • PixInsight Addict
  • ***
  • Posts: 100
    • Light Vortex Astronomy
Thank you guys - always appreciative of feedback! :)
Rob, thank you for your clarification. To be more correct, I have changed references of readout signal to sensor bias signal.
- Avalon M-Uno
- Takahashi FSQ-85ED, Altair Astro 8" RC with Astro-Physics CCDT67 Telecompressor
- QSI 660wsg-8, Starlight Xpress Lodestar X2
- Astrodon E-Series Gen2 LRGB 1.25", Astrodon HA, OIII & SII 3nm 1.25"

Offline pfile

  • PTeam Member
  • PixInsight Jedi Grand Master
  • ********
  • Posts: 4729
cool, great tutorial - it is wonderful to have clear explanations of this stuff out there.

rob

Offline kayronjm

  • PixInsight Addict
  • ***
  • Posts: 100
    • Light Vortex Astronomy
Hehe, thank you! :)

I have updated the screenshots on all my current tutorials, including this one, to display them at full-resolution as captured. It should be much easier to read settings on them now.
- Avalon M-Uno
- Takahashi FSQ-85ED, Altair Astro 8" RC with Astro-Physics CCDT67 Telecompressor
- QSI 660wsg-8, Starlight Xpress Lodestar X2
- Astrodon E-Series Gen2 LRGB 1.25", Astrodon HA, OIII & SII 3nm 1.25"

Offline MikeOates

  • PixInsight Addict
  • ***
  • Posts: 278
Kayron,

A great tutorial, pity it was not around when I was learning this stage. One point I would make concerns your last paragraph. I would not recommend anyone deletes any original files. The reason being, at some time in the future when you re-process them because you have learned new techniques or new processes are released, an example being SuperBias, it allows you to start again.

Mike

Offline kayronjm

  • PixInsight Addict
  • ***
  • Posts: 100
    • Light Vortex Astronomy
Kayron,

A great tutorial, pity it was not around when I was learning this stage. One point I would make concerns your last paragraph. I would not recommend anyone deletes any original files. The reason being, at some time in the future when you re-process them because you have learned new techniques or new processes are released, an example being SuperBias, it allows you to start again.

Mike

Thanks Mike! I agree with you. I was careful in the wording and said "you could" to accentuate the idea that this is a possibility, but not necessarily a recommendation. I personally keep everything as was captured, since harddisk space (internal and external) is cheap these days so it is easy to keep backups of everything. Some may prefer to calibrate everything and simply keep these calibrated light frames and that is all. It is entirely up to the user! :)
- Avalon M-Uno
- Takahashi FSQ-85ED, Altair Astro 8" RC with Astro-Physics CCDT67 Telecompressor
- QSI 660wsg-8, Starlight Xpress Lodestar X2
- Astrodon E-Series Gen2 LRGB 1.25", Astrodon HA, OIII & SII 3nm 1.25"

Offline cdavid

  • PixInsight Addict
  • ***
  • Posts: 137
    • Through Space and Time
Excellent Tutorial!

One comment.  It is usually not necessary to subtract a Master Dark from the Flats unless the Flat exposures are of significant time >1 second.
Simply calibrating with the Master or Super bias and then integrating to create the master flat suffices.

Carlos

Offline topboxman

  • PixInsight Addict
  • ***
  • Posts: 292
    • Peter's Gallery
I always thought Cosmetic Correction was equivalent of Bad Pixel Mapping and you could only do CC/BPM or dark subtraction but not both. The tutorial showed to use Cosmetic Corrected lights to calibrate with master dark as well as master bias and master flat. I would expect strange looking calibrated lights with darks after lights were Cosmetic Corrected.

If hot/bad pixels were removed from the lights using CC/BPM and dark subtraction was done next, won't it resulted black holes where hot/bad pixels were removed?

So, it's okay to do dark subtraction after Cosmetic Correction?

Peter

Offline pfile

  • PTeam Member
  • PixInsight Jedi Grand Master
  • ********
  • Posts: 4729
i would think that CC after dark subtraction makes sense, but not before for the reasons you point out.

rob

Offline topboxman

  • PixInsight Addict
  • ***
  • Posts: 292
    • Peter's Gallery
I would agree with you.

Peter

Offline kayronjm

  • PixInsight Addict
  • ***
  • Posts: 100
    • Light Vortex Astronomy
I agree on principle, but I have yet to see any of those dark pixels left over on the images I used in the tutorial's writing. I'm looking carefully but I'm not seeing any ill-effects.
- Avalon M-Uno
- Takahashi FSQ-85ED, Altair Astro 8" RC with Astro-Physics CCDT67 Telecompressor
- QSI 660wsg-8, Starlight Xpress Lodestar X2
- Astrodon E-Series Gen2 LRGB 1.25", Astrodon HA, OIII & SII 3nm 1.25"

Offline pfile

  • PTeam Member
  • PixInsight Jedi Grand Master
  • ********
  • Posts: 4729
it is possible that the order is not important if you're reasonably sure that you've only killed true hot/warm/cold pixels with the CC. of course, how can you ever know? doing calibration first is definitely safer, if only because it's by-the-book :)

rob

Offline topboxman

  • PixInsight Addict
  • ***
  • Posts: 292
    • Peter's Gallery
Hi kayronjm,

Maybe try do what Rob suggested and compare to your tutorial procedure to see if it makes any difference? Even a tiny difference can help.

Thanks for writing the tutorial.

Peter