Problem calibrating images captured in Nebulosity

[DanielF]

Hello, my name is Daniel Franzén and I'm trying to learn how to process images in PixInsight.

I've read Vicent's guide (http://pixinsight.com/tutorials/master-frames/en.html) on how to create master calibration frames and I've created a master bias frame and a master flat frame, calibrated with the master bias. No dark frame is used.

Since Neb uses (i think) the 0-65535 range I used the format hints lower-range 0 upper-range 65535 when calibrating the images.
All images where saved as 32 bit floating point (It sounded better but I've understood that 16bit unsigned is sufficient)

However when I try to calibrate a singe light frame with my calibration frames I get an error saying "Zero or insignificant scaling factor for flat frame channel# 0". I'm using the same input hints here as in calibration.
If I remove the input hints I don't get an error but the resulting calibrated image is destroyed (see attached image)

Any ideas what I'm doing wrong?
Regards
Daniel

[Philip de Louraille]

Look at your FIT files as they *came out* of Nebulosity (before feeding them to any other program) and look at the FIT headers.
THey ought to be 16-bit but do you have a BZERO field? If so, what is it?
If it is "0", then indeed your data goes from 0 to 65535 and is unsigned.
If it is 32768, then your data is signed and doing what you did below eliminated half your range and probably 90% of your data.
To avoid problems between Neb and PixInsight, always save your Nebulosity files as unsigned 16 bits FITS.

To another point, PI just came with a new script to make master frames (that is supposed to be a lot less complicated than the prior one.) I haven't played with it yet but it is found under Scripts and is named Batch Preprocessing. Give it a go.

[Geoff]

To another point, PI just came with a new script to make master frames (that is supposed to be a lot less complicated than the prior one.) I haven't played with it yet but it is found under Scripts and is named Batch Preprocessing. Give it a go.

I can verify that it works very well.  Harry has done a video to explain it, which is well worth looking at: http://www.harrysastroshed.com/pixinsighthome.html
Geoff

[DanielF]

Look at your FIT files as they *came out* of Nebulosity (before feeding them to any other program) and look at the FIT headers.
THey ought to be 16-bit but do you have a BZERO field? If so, what is it?
If it is "0", then indeed your data goes from 0 to 65535 and is unsigned.
If it is 32768, then your data is signed and doing what you did below eliminated half your range and probably 90% of your data.
To avoid problems between Neb and PixInsight, always save your Nebulosity files as unsigned 16 bits FITS.

To another point, PI just came with a new script to make master frames (that is supposed to be a lot less complicated than the prior one.) I haven't played with it yet but it is found under Scripts and is named Batch Preprocessing. Give it a go.

Hello.
This is what the FITS Header looks like in Nebulosity's FITS Header Tool

HDU 1: Image
Image has 2 axes and is 4770 x 3178 in size
HDU has 21 keywords:
SIMPLE  =                    T / file does conform to FITS standard
BITPIX  =                  -32 / number of bits per data pixel
NAXIS   =                    2 / number of data axes
NAXIS1  =                 4770 / length of data axis 1
NAXIS2  =                 3178 / length of data axis 2
EXTEND  =                    T / FITS dataset may contain extensions
COMMENT   FITS (Flexible Image Transport System) format is defined in 'Astronomy
COMMENT   and Astrophysics', volume 376, page 359; bibcode: 2001A&A...376..359H
CREATOR = 'Nebulosity v2.4.0'  / program and version that created this file
INSTRUME= 'Canon DSLR: Canon EOS 500D' / instrument name
DATE    = '2011-03-30 20:58:51' / UTC date that FITS file was created
DATE-OBS= '2011-03-30T20:53:33' / YYYY-MM-DDThh:mm:ss observation start, UT
EXPOSURE=                 300. / Exposure time s
GAIN    =                    4 / Camera gain
OFFSET  =                    0 / Camera offset
BIN_MODE=                    1 / Camera binning mode
ARRAY_TY=                    1 / CCD array type
XBINNING=                    1 / Camera binning mode
YBINNING=                    1 / Camera binning mode
EXPTIME =                 300. / Exposure time s
FILTER  = 'No filter'          / Optical filter name

I don't have a BZERO field. What does that mean then?

I've tried the preprocessing script but with no luck. That was with another set of files though and I will try it again with the same files that I used here and see what happens.

[DanielF]

I tried the preprocessing script now with the same images that I used during calibration in my first post.
It didn't work, the calibrated image came out like before.

I will try to convert my original fits to 16 bit unsigned integer and try again and see if that helps.

/Daniel

[Harry page]

Hi

Just a idea , have you unchecked the" optimise dark frame "

Harry

[DanielF]

Hi

Just a idea , have you unchecked the" optimise dark frame "

Harry

Hi Harry.
Yes the optimize dark frame was unchecked.

I converted all the images (well, the small subset I used for testing) to 16 bit and ran the preprocessing script again, and this time it came out ok!
Hopefully everything will work now after I convert the rest of my fits files to 16 bit.

Good times!

[topboxman]

Hi Daniel,

I also capture images with Nebulosity. I always capture in 16 bit format including darks, flats and bias with SXVR-M25C OSC camera. I never had good luck calibrating with PixInsight. I use Nebulosity's Bad Pixel Mapping using a stack of darks instead of dark subtraction because Sony CCD chip has very low noise and calibrating with dark frames is actually worse.

Anyway, I use Nebulosity "Pre-process multiple sets" to calibrate Bad Pixel Mapped lights with flats and bias. "Pre-process multiple sets" is very quick and works fine. All calibrated subs are outputted in 32 bit float. From then on I use PixInsight starting with DeBayering, Star Alignment, Image Integration and many post processing tools. The results turn out fine.

I am glad you got calibration working with PixInsight. Looking at your calibrated M51, it looks like you are using OSC camera due to Bayer Matrix. What camera are you using?

Peter

[DanielF]

Hi Daniel,

I also capture images with Nebulosity. I always capture in 16 bit format including darks, flats and bias with SXVR-M25C OSC camera. I never had good luck calibrating with PixInsight. I use Nebulosity's Bad Pixel Mapping using a stack of darks instead of dark subtraction because Sony CCD chip has very low noise and calibrating with dark frames is actually worse.

Anyway, I use Nebulosity "Pre-process multiple sets" to calibrate Bad Pixel Mapped lights with flats and bias. "Pre-process multiple sets" is very quick and works fine. All calibrated subs are outputted in 32 bit float. From then on I use PixInsight starting with DeBayering, Star Alignment, Image Integration and many post processing tools. The results turn out fine.

I am glad you got calibration working with PixInsight. Looking at your calibrated M51, it looks like you are using OSC camera due to Bayer Matrix. What camera are you using?

Peter

Hello Peter.
I've always used Nebulosity to pre-process and stack my images but I wanted to learn how to do it in PixInsight as well and compare the results. I get the feeling that you can customize and control calibration in PI at a higher level than in Nebulosity, but I don't know if the results are any better and that what counts in the end.
I also mostly use Bad Pixel Maps instead of darks since I don't have a cooled ccd-camera and don't like spending hours on collecting darks outside. (I live in Sweden and the winters can get pretty cold).

Your workflow seems like a good idea if Nebulosity does an equally good job at calibration since the "Pre-Process multiple sets" in Neb is way faster than pre-proccesing in PI.

The camera that I'm currently using is a Canon 500D Unmodded DSLR.

/Daniel

[DanielF]

Hello again.
Just wanted to show what I managed to do with my images after I got the calibration sorted.

After registration and integration I've applied DBE, color calibration, histogram transformation, ACDNR and saturation curve.
It turned out better than my previous attempt using Photoshop CS5.



/Daniel

[Philip de Louraille]

Very NICE! And excellent work!

[DanielF]

Very NICE! And excellent work!

Thank you very much!