Author Topic: Calibration and align batch operations  (Read 135805 times)

Offline Juan Conejero

  • PTeam Member
  • PixInsight Jedi Grand Master
  • ********
  • Posts: 7096
    • View Profile
    • http://pixinsight.com/
Re: Calibration and align batch operations
« Reply #270 on: 2012 April 10 11:25:33 »
Quote
Will you marry me?

You already married me when you bought your PixInsight license  >:D

Quote
I am not sure that rejection maps for master images are really necessary.

For master bias and dark frames, probably not. For the master flat frame it may be necessary, especially for sky flats (rejection of stars). Perhaps we should have two independent options for generation of rejection maps, one for lights and the other for calibration frames?
Juan Conejero
PixInsight Development Team
http://pixinsight.com/

Offline Josh Lake

  • PixInsight Old Hand
  • ****
  • Posts: 424
    • View Profile
Re: Calibration and align batch operations
« Reply #271 on: 2012 April 10 11:39:13 »
I am right there with you all, trying this amazing piece of work as it gets hammered out. Fantastic, just fantastic, so much easier to set up a run and then analyze the results.

I had an OSX bug on the last version (will check this one shortly) where the Output Path changed into a clone of the Image Registration Reference path after a dialog box was worked with.

This will be fixed soon with a new version of the ImageCalibration module.

Looking forward to this new version. A while back, Vicent mentioned something about using Multiscale within ImageIntegration to take care of moving dust donuts. Any chance that this will be in there? I have some very challenging frames that flats are having a hard time with.

Edit: Ah, this new version will not work on Macs because Cosmetic Correction is not compiled for OSX yet, right?

Offline Juan Conejero

  • PTeam Member
  • PixInsight Jedi Grand Master
  • ********
  • Posts: 7096
    • View Profile
    • http://pixinsight.com/
Re: Calibration and align batch operations
« Reply #272 on: 2012 April 10 11:41:14 »
Quote
I still can not select the same file twice ( i.e master bias instead of darks )

I don't understand why you want to do this, so please clarify. If you don't use dark frames, simply don't select any dark frame and the script will generate and apply just a master bias. Or is there something else that I am missing?

One of the purposes of this script is to simplify some tasks by making it very difficult to make mistakes. Duplicate frames and incorrectly selected frames are two common pitfalls. I don't see any real advantage in allowing them.

Quote
still hate I have to mess about with file names

One of the key elements of any good data reduction procedure is keeping all the data well organized. The script tries to enforce the best possible organization and file structure. Correct file names and frame types stored in FITS headers are essential items; this is a very basic requirement for any decent image acquisition software IMO.

With the latest version of the script you just have to be a bit careful to name your images: just include the 'bias', 'dark', 'flat' and 'light' bits as appropriate in the file names and everything should work automatically.

Quote
Small bug -the scroll bar on the consol disapears after running the script

Yes, I have seen this. It is a small bug in the PI core application, not in the script. I'll fix it in the next core update.
Juan Conejero
PixInsight Development Team
http://pixinsight.com/

Offline Harry page

  • PTeam Member
  • PixInsight Jedi Knight
  • *****
  • Posts: 1457
    • View Profile
    • http://www.harrysastroshed.com
Re: Calibration and align batch operations
« Reply #273 on: 2012 April 10 12:15:51 »
Hi

I did not know about not needing to select a dark if not needed  :-* the warning message put me off  :laugh: so I will stop winging about this  :-[ ;D  but info in the warning might help like " No Dark frames selected If you continue a master bias will be used instead "  Or the like

But the bit about having the word " Flat" in the file name does not work though  >:D   

Regards Harry
Harry Page

Offline Josh Lake

  • PixInsight Old Hand
  • ****
  • Posts: 424
    • View Profile
Re: Calibration and align batch operations
« Reply #274 on: 2012 April 10 12:29:45 »
With the latest version of the script you just have to be a bit careful to name your images: just include the 'bias', 'dark', 'flat' and 'light' bits as appropriate in the file names and everything should work automatically.

Out of curiosity, how does the script handle weird filenames with multiple filter labels in them? It's bad practice, but I'm working with a guy who gets filenames like this out of his automated observatory software:

GreenFilter_Red001.fits    or    DarkFrame_Bias7.fits

It drives me crazy while organizing and I just end up batch renaming all the files anyway, but if this script will pull filename and/or FITS header info, I can just load these messy files into the hopper and let the script organize them, yes? It's bad practice, but it'll save me a lot of time when working with his files.

Offline mmirot

  • PixInsight Padawan
  • ****
  • Posts: 881
    • View Profile
Re: Calibration and align batch operations
« Reply #275 on: 2012 April 10 12:34:50 »
This happens because your light frames have the keyword FILTER=Ha1.00 in their headers, but your flat frames don't. The script tries to match sets of images acquired with the same filters.

The solution is easy though: add your flat frames as custom files, and specify: image type = flat, filter = Ha1.00. It works fine this way (I've checked).

Ahhhh. Great.

If I understand this correctly,  I could using this match to flats for the meridian flip with a rotator etc????

Max



Offline bitli

  • PTeam Member
  • PixInsight Guru
  • ****
  • Posts: 512
    • View Profile
Re: Calibration and align batch operations
« Reply #276 on: 2012 April 11 00:32:53 »
Just one think:

Thank you very much for this script.

And congratulation to the new weds  :)

-- bitli

Offline Juan Conejero

  • PTeam Member
  • PixInsight Jedi Grand Master
  • ********
  • Posts: 7096
    • View Profile
    • http://pixinsight.com/
Re: Calibration and align batch operations
« Reply #277 on: 2012 April 11 04:29:41 »
Quote
And congratulation to the new weds

Well, this script is more to prevent divorce before marriage. Some bridegrooms were getting a bit 'aggressive'...  ::)

Thanks! :)
Juan Conejero
PixInsight Development Team
http://pixinsight.com/

Offline mmirot

  • PixInsight Padawan
  • ****
  • Posts: 881
    • View Profile
Re: Calibration and align batch operations
« Reply #278 on: 2012 April 11 07:13:30 »
Boy I must be divorced.

I can't even get a response out of Juan.  :sad:  :sad:

No respect. No respect at all.  ::)

Max


Offline Juan Conejero

  • PTeam Member
  • PixInsight Jedi Grand Master
  • ********
  • Posts: 7096
    • View Profile
    • http://pixinsight.com/
Re: Calibration and align batch operations
« Reply #279 on: 2012 April 11 07:40:11 »
Quote
No respect. No respect at all.  ::)

 :angel:

Quote
I could using this match to flats for the meridian flip with a rotator etc?

I think so. Just use a 'virtual filter' of your choice to match flipped flats with flipped lights added as custom frames. Should work ;)

Quote
Cosmetic correction works great but I don't see why this is greyed out when calibrate only is checked. It may be desirable to produce calibrated and Cal_CC images only at times.

Agreed. I'll see how to change this in the next version.

Quote
Also, I split my work into two script runs since I had multple nights. The weather got warmer so the lasts few nights were performed at -30C  rathar than -35.

Why? Unless there are significant seeing differences (in which case the first thing to do is to remove bad frames from the whole set), out dark frame optimization routine will scale your darks automatically. So I think you can calibrate/align/integrate the whole set without problems (and you'll get a higher SNR improvement by integrating more frames in the same stack).
Juan Conejero
PixInsight Development Team
http://pixinsight.com/

Offline mmirot

  • PixInsight Padawan
  • ****
  • Posts: 881
    • View Profile
Re: Calibration and align batch operations
« Reply #280 on: 2012 April 11 09:28:45 »
Thanks Juan,

It all sounds great.  Looks like Kai and you made my preprocess pipeline dreams come true.
 :)

Quote
Also, I split my work into two script runs since I had multple nights. The weather got warmer so the lasts few nights were performed at -30C  rathar than -35.

Why? Unless there are significant seeing differences (in which case the first thing to do is to remove bad frames from the whole set), out dark frame optimization routine will scale your darks automatically. So I think you can calibrate/align/integrate the whole set without problems (and you'll get a higher SNR improvement by integrating more frames in the same stack).
[/quote]

I guess I don't understand the optimization routine fully. I aways thought that darks and lights of the same temp were more accurate to scale rather matching different temps,  -30 and -35 etc.

On my current camera it accumlates more 5-10x radioactive/ray hits due a kodak manufacting problem with the coverslip. Otherwise I might just take a long exposure a each temp.

I usually take a set of 25 darks at the same exposure and temp. as the lights. I am doing to much work?

(the original comment was about retaining paths I think?)

Max

Offline sleshin

  • PixInsight Old Hand
  • ****
  • Posts: 431
    • View Profile
Re: Calibration and align batch operations
« Reply #281 on: 2012 April 14 15:41:59 »
Just tried latest version and when "calibrate only" is checked, light frames are not calibrated. Loaded Bias, Darks, Flats, and uncalibrated Lights. The flats were calibrated and the master Bias, Darks, and Flats were properly done. But the script stopped there and didn't calibrate the lights. With "Calibrate Only" unchecked all works fine. Lights are calibrated and registered. Didn't enable Cosmetic Correction or Image Integration so can't comment on that.

Steve
Steve Leshin

Stargazer Observatory
Sedona, Arizona

Offline kwiechen

  • PixInsight Addict
  • ***
  • Posts: 186
    • View Profile
Re: Calibration and align batch operations
« Reply #282 on: 2012 April 15 06:00:40 »
minor problem

lines 650, 651 (engine.doLight)

if ( this.calibrateOnly )
      return;

should  be moved in front of line 744

... move here ...

      var SA = new StarAlignment;

and the CalibrateOnly checkbox works


Kai



Offline sleshin

  • PixInsight Old Hand
  • ****
  • Posts: 431
    • View Profile
Re: Calibration and align batch operations
« Reply #283 on: 2012 April 15 07:30:25 »
Thanks Kai. Is a corrected version of the script available somewhere?

Stevem
Steve Leshin

Stargazer Observatory
Sedona, Arizona

Offline Josh Lake

  • PixInsight Old Hand
  • ****
  • Posts: 424
    • View Profile
Re: Calibration and align batch operations
« Reply #284 on: 2012 April 15 09:33:00 »
One *very* minor bug -- a misspelling of 'light' as 'lihgt' (or similar) in one of the error messages (if you don't have flat frames selected to calibrate a certain filter of light frames).