New tutorial: Master Calibration Frames: Acquisition and Processing

[Niall Saunders]

That's what I thought. I imagine 0.97-0.99 should be optimal on the high end.

Do you save the outputs as 16 bit unsigned interger? 

Max

Hi Max - no, the most important thing is to 'acquire' your raw subs in 16-bit FITS integer (unsigned) mode. I haven't had a chance to play with the new ImgCal module yet, but I would very much doubt if it will work AT ALL if you try and feed it 32-bit Float images (for all the reasons that I have tried to explain elsewhere).

Once you have started to create the various intermediate (master) calibration frames, all of these can be saved in 32-bit Float mode - as PI 'understands' this environment, and will have internally adjusted the 'raw' data to 'fit' a [0.0, 1.0] floating-point range. So, you can work in 32-bit Float mode thereafter, and enjoy the fact that you won't have to worry about 'round-off' errors in your data.

Obviously, once you are ready to save in JPEG (for the web, etc.) you can export your final image to this 8-bit Unsigned Integer environment - but you will have saved a 'master' image in 32-bit Float anyway (wont you )

Cheers,

[mmirot]

I think a problem might be if you created the Masters as 32 bit then tries to apply them in the calibration module to acquired 16 bit unsigned images. 

I have not checked this yet.

Perahps PI just handle the 32 bit to 16 bit scaling internally ?

Max

[vicent_peris]

I think a problem might be if you created the Masters as 32 bit then tries to apply them in the calibration module to acquired 16 bit unsigned images. 

I have not checked this yet.

Perahps PI just handle the 32 bit to 16 bit scaling internally ?

Max

No problem with this if you generated your 32 bit files in PI. For PI, black is 0 and white 1 in 32 bit fp, or 0 and 65535 in 16 bit int. I always save my masters in 32 bit fp.

V.

[mmirot]

Ok,

That's what I hoped for. I have been saving them as 16 bits just in case.

Max

[Niall Saunders]

That's what I hoped for. I have been saving them as 16 bits just in case.

What it boils down to is that, for as long as we are all using "16-bit" cameras, the ONLY time that we need to worry about saving in 16-bit Unsigned Integer mode is when we are actually 'capturing' our raw data. Thereafter, once PI starts working with that 16-bit data, it is quite acceptable to save in 32-bit Float mode. OK, there is a very slight penalty in terms of increased HDD storage space, but the benefits are worthwhile.

And, remember, the reason PixInsight 'insists' on 16-bit UI 'raw' data is that this is the only data format that does not have to be 'translated' in order to be properly used by an image processing program. And, by that, I mean that a program would need to know a whole load of 'extra unique information' about a FITS image in order to be able to correctly make use of it. Many other programs do just exactly this - but that requires constant code revision  in order to be able to keep up with new hardware (which is what happens).

Stick to 16-bit UI at time of capture, or use a very simple ImageContainer/ScreenFormatConversion process to quickly change all your 32-bit F data into 16-bit UI and all will be well (that's how I work now, and it really takes just a few minutes after each capture session - after which I delete the 32-bit F 'raw' images forever).

Cheers,

[Jack Harvey]

Want to cliffy my interpretation of this tool tutorial.  Once you are ready to calibrate a stack of light frames and have you Bias, Dark and appropriate Flat Masters plugged into the slots on the ImageCalibration tool you check the Calibrate boxes under Bias and Darks, but NOT flats??

[Niall Saunders]

As I understand things Jack (and I am just about to sit down to my 'first run through'),

You start off with raw Lights, Darks, Flats and Bias frames

The first two steps are both ImageIntegration (II) steps, and these allow you to create a MasterDark and a MasterBias frames from your raw subs - you cannot, at this stage, create the MasterFlat because you will need BOTH the MasterDark and the MasterBias frame to pre-calibrate your Flats.

The next stage is to then use the ImageCalibration (IC) process, along with your new MasterDark and MasterBias frames, to calibrate the INDIVIDUAL Flat frames

Then you need to use II again, this time to integrate (stack) your newly calibrated Flat frames into a MasterFlat.

Only now do you have the three (master) calibration frames, which can now be applied to your Lights - again using the new IC process. as far as the tutorial then goes, you do NOT tick the 'calibrate' check box in the 'Master Flat' sub-section. The reason being that you have already just 'calibrated' the Flats.

The bit that I don't yet understand is why - at the time we are about to 'calibrate' the Lights - do we still then have the 'calibrate' box ticked for the 'Master Bias' and the 'Master Dark' sections

I need to go and 'have a play' - and see if I can work out the logic.

Cheers,

[Jack Harvey]

You see precisely my point.  Why have the Bias and Dark boxes checked and not the Flats?

[harist]

Hi all,

There is definetly some confusion there. Judging by fig.2 of the tutorial where 'Calibrate' is ticked for both MasterDark and MasteBias when calibrating Flat frames I also kept them ticked for the light frames (I created MasterDark, MasterBias, and MasterFlat frames as described in the tutorial, they all look nice). However the calibrated light frames are unusable (most details are gone only a few stars shown and even these can not be detected by StarAlignment which will not run). I have tried the same set (darks, flats, bias, lights) in ImagePlus and they work OK. Apparently I am doing something wrong due to the OSC nature of my files (Starlight Xpress SXVR-M25C camera). Would I need to debayer OSC files (darks, flats, bias, lights) before using them in the IC tool?

Thanks for any suggestions,

Tasos

[Nocturnal]

Tasos,

you want to debayer before registering and stacking. So calibration (offset subtraction and flattening) should be done on raw frames. To be honest if you have a windows box you're better of using DeepSkyStacker. If you want to stack in PI you could use DSS to create calibrated subs for you.

[harist]

Thanks Sander, that was quick! So far I have been calibrating and debayering in ImagePlus (which calibrates raw frames and then debayers as you say), then registering and stacking and experimenting ... in PI. I thought IC could handle OSC files.

Thanks again,
Tasos

[Jack Harvey]

WOrking on some data I had I ran the light frame calibration with all three checkboxes checked (BIAS,DARK and FlAT) and then with FLAT unchecked.  To my eye the FLAT unchecked as stated in the tutorial is the way to go.

[Nocturnal]

Tasos, PI has a debayer module (I wrote it ) but you need to apply it at the right moment.

[harist]

Sander, I did apply debayer to the calibrated frames (not before), but as I said the calibrated frames were problematic before debayering(most of the info was gone). Also applying debayer to them reduced their size! I tried debayer on the uncalibrated frames and it worked OK (thus the calibration tool affects the OSC channels somehow).

Thanks,
Tasos

[Nocturnal]

Tasos,

sounds like your calibration isn't working properly. Most likely you're subtracting too much data. Verify that you're subtracting bias only 1 time. In other words, if you subtract darks, don't also subtract bias from your lights.

You must have used superpixel debayer. Try bilinear if you want to keep the same resolution.