Calibration and align batch operations

[Juan Conejero]

Will you marry me?

You already married me when you bought your PixInsight license 

I am not sure that rejection maps for master images are really necessary.

For master bias and dark frames, probably not. For the master flat frame it may be necessary, especially for sky flats (rejection of stars). Perhaps we should have two independent options for generation of rejection maps, one for lights and the other for calibration frames?

[Josh Lake]

I am right there with you all, trying this amazing piece of work as it gets hammered out. Fantastic, just fantastic, so much easier to set up a run and then analyze the results.

I had an OSX bug on the last version (will check this one shortly) where the Output Path changed into a clone of the Image Registration Reference path after a dialog box was worked with.

This will be fixed soon with a new version of the ImageCalibration module.

Looking forward to this new version. A while back, Vicent mentioned something about using Multiscale within ImageIntegration to take care of moving dust donuts. Any chance that this will be in there? I have some very challenging frames that flats are having a hard time with.

Edit: Ah, this new version will not work on Macs because Cosmetic Correction is not compiled for OSX yet, right?

[Juan Conejero]

I still can not select the same file twice ( i.e master bias instead of darks )

I don't understand why you want to do this, so please clarify. If you don't use dark frames, simply don't select any dark frame and the script will generate and apply just a master bias. Or is there something else that I am missing?

One of the purposes of this script is to simplify some tasks by making it very difficult to make mistakes. Duplicate frames and incorrectly selected frames are two common pitfalls. I don't see any real advantage in allowing them.

still hate I have to mess about with file names

One of the key elements of any good data reduction procedure is keeping all the data well organized. The script tries to enforce the best possible organization and file structure. Correct file names and frame types stored in FITS headers are essential items; this is a very basic requirement for any decent image acquisition software IMO.

With the latest version of the script you just have to be a bit careful to name your images: just include the 'bias', 'dark', 'flat' and 'light' bits as appropriate in the file names and everything should work automatically.

Small bug -the scroll bar on the consol disapears after running the script

Yes, I have seen this. It is a small bug in the PI core application, not in the script. I'll fix it in the next core update.

[Harry page]

Hi

I did not know about not needing to select a dark if not needed  the warning message put me off  so I will stop winging about this    but info in the warning might help like " No Dark frames selected If you continue a master bias will be used instead "  Or the like

But the bit about having the word " Flat" in the file name does not work though     

Regards Harry

[Josh Lake]

With the latest version of the script you just have to be a bit careful to name your images: just include the 'bias', 'dark', 'flat' and 'light' bits as appropriate in the file names and everything should work automatically.

Out of curiosity, how does the script handle weird filenames with multiple filter labels in them? It's bad practice, but I'm working with a guy who gets filenames like this out of his automated observatory software:

GreenFilter_Red001.fits    or    DarkFrame_Bias7.fits

It drives me crazy while organizing and I just end up batch renaming all the files anyway, but if this script will pull filename and/or FITS header info, I can just load these messy files into the hopper and let the script organize them, yes? It's bad practice, but it'll save me a lot of time when working with his files.

[mmirot]

This happens because your light frames have the keyword FILTER=Ha1.00 in their headers, but your flat frames don't. The script tries to match sets of images acquired with the same filters.

The solution is easy though: add your flat frames as custom files, and specify: image type = flat, filter = Ha1.00. It works fine this way (I've checked).

Ahhhh. Great.

If I understand this correctly,  I could using this match to flats for the meridian flip with a rotator etc?

Max

[bitli]

Just one think:

Thank you very much for this script.

And congratulation to the new weds 

-- bitli

[Juan Conejero]

And congratulation to the new weds

Well, this script is more to prevent divorce before marriage. Some bridegrooms were getting a bit 'aggressive'... 

Thanks!

[mmirot]

Boy I must be divorced.

I can't even get a response out of Juan.   

No respect. No respect at all. 

Max

[Juan Conejero]

No respect. No respect at all. 

 

I could using this match to flats for the meridian flip with a rotator etc?

I think so. Just use a 'virtual filter' of your choice to match flipped flats with flipped lights added as custom frames. Should work

Cosmetic correction works great but I don't see why this is greyed out when calibrate only is checked. It may be desirable to produce calibrated and Cal_CC images only at times.

Agreed. I'll see how to change this in the next version.

Also, I split my work into two script runs since I had multple nights. The weather got warmer so the lasts few nights were performed at -30C  rathar than -35.

Why? Unless there are significant seeing differences (in which case the first thing to do is to remove bad frames from the whole set), out dark frame optimization routine will scale your darks automatically. So I think you can calibrate/align/integrate the whole set without problems (and you'll get a higher SNR improvement by integrating more frames in the same stack).

[mmirot]

Thanks Juan,

It all sounds great.  Looks like Kai and you made my preprocess pipeline dreams come true.
 

Also, I split my work into two script runs since I had multple nights. The weather got warmer so the lasts few nights were performed at -30C  rathar than -35.

Why? Unless there are significant seeing differences (in which case the first thing to do is to remove bad frames from the whole set), out dark frame optimization routine will scale your darks automatically. So I think you can calibrate/align/integrate the whole set without problems (and you'll get a higher SNR improvement by integrating more frames in the same stack).
[/quote]

I guess I don't understand the optimization routine fully. I aways thought that darks and lights of the same temp were more accurate to scale rather matching different temps,  -30 and -35 etc.

On my current camera it accumlates more 5-10x radioactive/ray hits due a kodak manufacting problem with the coverslip. Otherwise I might just take a long exposure a each temp.

I usually take a set of 25 darks at the same exposure and temp. as the lights. I am doing to much work?

(the original comment was about retaining paths I think?)

Max

[sleshin]

Just tried latest version and when "calibrate only" is checked, light frames are not calibrated. Loaded Bias, Darks, Flats, and uncalibrated Lights. The flats were calibrated and the master Bias, Darks, and Flats were properly done. But the script stopped there and didn't calibrate the lights. With "Calibrate Only" unchecked all works fine. Lights are calibrated and registered. Didn't enable Cosmetic Correction or Image Integration so can't comment on that.

Steve

[kwiechen]

minor problem

lines 650, 651 (engine.doLight)

if ( this.calibrateOnly )
      return;

should  be moved in front of line 744

... move here ...

      var SA = new StarAlignment;

and the CalibrateOnly checkbox works


Kai

[sleshin]

Thanks Kai. Is a corrected version of the script available somewhere?

Stevem

[Josh Lake]

One *very* minor bug -- a misspelling of 'light' as 'lihgt' (or similar) in one of the error messages (if you don't have flat frames selected to calibrate a certain filter of light frames).