Using iTelescope calibration files in BatchPreprocessing

[dayers]

Winter is coming, so I have moved indoors to collect data using iTelescope. They make a calibration library available, and I am trying to adapt their master files to work within PI, specifically in BP. I think I need to re-scale the calibration files to the 0-1 range. It seems like I have done this before, but my ol' brain won't dredge up the answer.

I know that iTelescope supplies pre-calibrated image files, but I would like to make a comparison with the BP result.

I would appreciate a kick-start here. Do I edit the FITS header? And when I do, how do I save it back into the active image's FITS header?

Thanks.

Dave, befuddled.

[jerryyyyy]

Will be interested in the reply.  I signed up a few months ago and have processed one image using their processing.  Looked tough to do it yourself. 

[pfile]

i think jim has a pretty good handle on this stuff and probably has a post somewhere here describing what to do. the main problem is that they have most likely produced their calibration masters with maxim and the sample format is something that PI is not expecting. so you are right, the files need to be normalized properly to the [0,1] range. generally if you have the "ask if out-of-range" box ticked in the FITS reader, when you open one of the calibration masters PI will tell you what the smallest sample and the largest sample in the file are. you can use those as the input min/max limits. but i think others have devised a more robust way of doing this using pixelmath.

rob

[bitli]

Hi, I would be interested by any result too.  Currently I use pre-calibrated files from iTelescope, as trying to redo calibration was too time consuming and without clear benefits, but I did not try manual calibration of their images since a long time.

iTelescope provides both raw bias/dark/flats and master bias/dark/flat. You could use one or the other (even mix and match in theory).
 
I assume that you want to start from the raw files if you want to use BPP. This should not be a problem, normally the raw files are just I16 files in the same format as the image file.  Currently I have no problem reading the images from Itelescope, and there is no need to do stuff with the FITS headers (maybe different location/scopes are not setup exactly in the same way). 

You could also start from the master bias/dark/flat to calibrate your images. I found this more difficult as it is not clear if the bias was already removed from the darks/flats for example. This could be find by careful examination of the header. The iTelescope master images are also 16 bits unsigned, the one produced by PI as 32 bits (24 bits of precision, really), but PI is pretty happy reading 16 bits images. You may have to tell BPP which file is a bias/dark/flat if it does not recognize the keyword, but at least on a recently downloaded master dark there is an IMAGETYP.

I used to have problems with the PEDESTAL keyword, the value of this keyword is interpreted differently between PI and some other software. This may cause truncation of low values and 'no correlation' errors during calibration, so if you have this type of error you may have look at the FITS header and around the forum for details.

Bottom line, go ahead using the raw files as is and tell us if you have any problem (or any success).
-- bitli

[dayers]

Thanks for your responses. I just downloaded a parcel of raw calibration files to go with a series of NB lights I recently exposed via iTelescope T11 in Mayhill, NM. I'll pour them into BP and let you know how it goes. I'm really not sweating this too much, because I have always had good results using the pre-calibrated files, but who knows?

Dave

[dayers]

OK, here is what I did:

Set up BatchPreprocessing with 20 each bias, darks, flats, using raw files as supplied by iTelescope. Aligned and integrated the result (18 files) for a comparison image.

Did it again, except using the master bias, darks, flats generated from above.

Compared with aligned and integrated pre-calibrated files. The results were identical, with essentially identical image median values.

Conclusion:  Only the Maxim-generated master bias, darks, flats should not be used with BatchPreprocessing. Use raw calibration files from iTelescope's calibration library or the masters generated by BP. I will be content from now on with using the pre-calibrated files, since brewing my own seems to offer no advantage.

Dave

[bitli]

Note that iTelescope flip/rotates the images that are taken when the pier is on the east side - BOTH FOR CALIBRATED AND UNCALIBRATED IMAGES ! at least on some telescopes, but I assume that this is true on all.
If you do your own calibration, you have to figure out how to turn the masters respectively to each image depending on the pier side - really an error prone manual process. And I did not even check if there was not a flip on the flats or others masters.... So indeed use calibrated image (I had different but similar problems with other providers of remote images).

Even when you operate on the calibrated images you have to be careful.  You cannot blindly remove a dead column on a sequence of images, unless they were all taken on the same pier side. You must process the east side together and the west side together.
-- bitli

[dayers]

Thanks, bitli, that is very helpful information. I will check out the telescope I use a lot, T-11 at New Mexico. You confirmed my conclusion that usually using supplied calibrated frames is the best (and easiest) way to go.

Is it safe to mix east-side and west-side frames and let StarAlignment and ImageIntegration sort it out? It seems like I have done this in the past and didn't have problems.

Dave

[bitli]

Is it safe to mix east-side and west-side frames and let StarAlignment and ImageIntegration sort it out? It seems like I have done this in the past and didn't have problems.

Yes, no problem.  If you look at the images, you see that they have all the same orientation, which cause no problem for PI (or any other software). Normally, with a German mount, you should expect that the images taken on one side of the pier have a 'rotated' orientation from images taken on the other side. PixInsight would be happy with that too and orient them automatically. Note that you usually take most images on one side of the pier anyhow, at least on a single night.

You cannot use dark, bias , flats or uncalibrated images without thinking. You cannot reference a dark in CosmeticCorrection without thinking.

If you use calibrated images only you have no problem at all UNLESS you make operations based on the sensor orientation (Cosmetic Correction with explicit row/columns is the only one I can think of). Using automatic Cosmetic Correction is no problem.  Using it with a dark to find hot pixel is a no-no.

-- bitli