Newbie question on batch integration of calibration files

[Buzz]

Hi, I have been using Nebulosity and Maxim for a number of years and have just moved over to PixInsight.  I'm creating new master calibration files from all my saved exposures. The tutorials suggest I have to do each one separately. They were all acquired in Maxim, is there an equivalent script that will batch process all my files at once, using the FITs header to distinguish temperature, type, time and filter?

By the way, nice place Valencia. I used to work in Almussafes

[Geoff]

See here http://www.harrysastroshed.com/preprocessing.html
Geoff

[Buzz]

Thanks Geoff -  I checked out the video but I'm at the stage beforehand, creating master flats, biases and darks.

In Maxim, it reads the FITS headers and sort the files according to type, equipment, filter and exposure. It integrates and creates master flats, biases and darks.

My question is can I do this in Pixinsight or do I have to load each set of calibration exposures manually and create a master, one by one. (I have four scopes, 7 filters and a range of standard exposure times from 15s to 2400s.  Any ideas?

[pfile]

the BatchPreprocessing script was designed to handle cases like yours... try it out.

rob

[Buzz]

Hi - I'm really struggling with this. I have used Maxim and Nebulosity in the past and not had issues - same raw files for biases, darks, flats and lights in PI and my final image is black apart from a few pinpricks of the brightest stars which are just off black. It really is black, even after histogram adjustments.

This is becoming frustrating. I mean, transputers and OCCAM were easier than this. I have looked at the various tutorials and videos and there is nothing I'm doing differently as far as I know.

It does not matter if I use my Maxim Masters or generate new PI ones, same result. This has to be something obvious. Any ideas?

[NGC7789]

Sounds like you are not using the Screen Transfer Function.

[pfile]

at this point you've got to post some images; start with the stack, if that looks bad then maybe a single bias, dark, flat and light.

rob

[Buzz]

thanks, I will create some files.  By the way, I am using STF and also checking the FITS files with Maxim and Nebulosity just to make sure.  In this forum, what is the protocol for posting images...is it within the message, URL link or is there a separate file section?

[pfile]

there is a fileserver, called "endor", which you can use, but you have to PM Juan to get an account on that server.

otherwise you can just use dropbox or google drive or whatever and just post the URL.

FITS is not supported by forum attachments, and the size is limited too much, so that's not an option…

rob

[Harry page]

Hi

What file format are you saving in i.e Save it as a 16-bit unsigned integer or you might get some problems

Harry

[Buzz]

Hi Folks.  The links are on my webpage below. I was using 32-bit unsigned. (Nebulosity cannot read them)

I used the integration tool to combine the darks and biases, as per the PI tutorial.
I also used the tutorial setting to generate a master flat file - but when I used the Batch Pre-Processor to calibrate and integrate lights, it would not recognize this as a flat file. I then tried using the Batch Pre Processor itself with my master bias and dark to create the Master Flat from all my flat files. This file was recognized by the Batch Pre Processor as a flat file OK. I then used the Batch Pre-Processor on a simple four exposure light file set of the same filter, temperature and exposure as my master files- the individual calibrated images and their integrated version were the same, black with star dots.

The master bias, darks and flats look OK in PI

url:    http://www.toinfinityandbeyond.me.uk/astronomy/Files.html

[Phil Leigh]

Hi can you post a link to 3 unprocessed raw light frames? Then we can try processing them...
thanks
Phil

[pfile]

i've had problems trying to use masters created outside of BPP in BPP. you have to be careful that your fits reader direction is set up the same way that BPP sets it up. also the masters created outside BPP are missing the FITS keywords to make them recognizable as flats for a particular filter. when that happens you need to use the "add custom frame" functionality in BPP to force BPP to realize what filter the flats are for.

you can also edit the FITS header of the masters to put in the right keywords such that BPP can recognize them automatically.

rob

[Buzz]

OK - thanks for the pointer on FITS header files. That is easy to fix on a few files.

I have zipped up four unprocessed subs from Maxim on the same URL page.

http://www.toinfinityandbeyond.me.uk/astronomy/Files.html

All the calibration images and lights were taken with Maxim DL. I processed these in Maxim DL without issue, so I will be interested to know what has gone wrong here. Note, I let PI save my processed files as 32 bit float. One other poster suggested it should be 16-bit unsigned. Could that be the reason - seems strange if it is.
regards
Chris

[Harry page]

Hi

When processing in pixinsight 32 bit is the way to go , but when bringing files from maxim to pixinsight 16 bit is what you want to do


Harry