Author Topic: Calibration and align batch operations  (Read 135822 times)

Offline georg.viehoever

  • PTeam Member
  • PixInsight Jedi Master
  • ******
  • Posts: 2132
    • View Profile
Re: Calibration and align batch operations
« Reply #315 on: 2012 April 27 06:45:58 »
I generally determine the "real" hot pixels by doing a cosmetic correction on my masterDark. I later apply the same parameters to lights.
Georg
Georg (6 inch Newton, unmodified Canon EOS40D+80D, unguided EQ5 mount)

Offline jeffweiss9

  • PixInsight Old Hand
  • ****
  • Posts: 339
    • View Profile
Re: Calibration and align batch operations
« Reply #316 on: 2012 April 27 08:17:00 »
Thanks, Georg-
   That's what I thought ought to be the way to do it.  I didn't mention that I use library darks (I'm a field imager; no obs) that typically are up to a few months old so I don't expect the hot pixels to match perfectly.  However, in this case of imaging on a (freakishly) warm night, I went to a new temperature for at least some of the data and took darks for that temperature the next night at home.  I didn't notice any difference in this problem for those subs with 'fresh' darks, however, compared to the ones with 2 month old library darks.
   But I'll go back to getting the settings from my MasterDarks.  I think I was diverted from that because AutoDetect gave what looked like more complete elimination of hot pixels in the test sub but, unlike the manual entry, autodetect doesn't show the quantity of hot pixels its removing, which presumably was much larger than it should have been.
   Other suggestions still appreciated.
Oh yes and thanks to KWA for authoring this very useful PI enhancement.
-Jeff
« Last Edit: 2012 April 27 08:26:11 by jeffweiss9 »
APM LZOS 130/780 f/6 LW CNC II APO, Riccardi 1.0 FF or 0.75 FF/FR, Tak EM-200 Temma2, FLI Microline ML-16200, Astrodon E Gen 2 filters and 5nm Ha, Orion 50mm Guider & Starlight Xpress Lodestar X2.

Offline cs_pixinsight

  • PixInsight Addict
  • ***
  • Posts: 156
    • View Profile
Re: Calibration and align batch operations
« Reply #317 on: 2012 April 27 09:19:29 »
Yesterday I also noticed the new multi-image master dark files being generated by this script are not working with CosmeticCorrection.  The module simply errors out when trying to load it - I'm sure confused by the extra images stored within it. 

My work around for now is to open the master dark in PI and save it under a new name, then use CC with this new file.

But, I'm wondering if the BatchProcessing script should save all the rejection maps and masters as individual files instead of the single multi-image one that is currently being generated.  Or, should CosmeticCorrection be updated to interpret both file types?

Craig

Offline Harry page

  • PTeam Member
  • PixInsight Jedi Knight
  • *****
  • Posts: 1457
    • View Profile
    • http://www.harrysastroshed.com
Re: Calibration and align batch operations
« Reply #318 on: 2012 April 27 09:38:56 »
Hi
In my vid I used the master dark as the Map and previewed it on a image so I could make sure I was not removing to much i.e less is more  :D

We also use the words "Hot pixels" to much , most chips have very few true hot pixels and most are warm Pixels and therefore How low do you go with

the sigma setting before it causes other defects ;D

Harry
Harry Page

Offline Juan Conejero

  • PTeam Member
  • PixInsight Jedi Grand Master
  • ********
  • Posts: 7096
    • View Profile
    • http://pixinsight.com/
Re: Calibration and align batch operations
« Reply #319 on: 2012 April 27 10:34:23 »
Juan are you going to give this script a push , would be well worth your time  ;D

My maximum priorities now are PI Core version 1.8 and PCL version 2.0. I am working like crazy on these. PI 1.8 has to be released and fully stabilized before Summer.

I think this script is right now pretty stable and reasonably feature complete, isn't it? Nice video by the way ;)
Juan Conejero
PixInsight Development Team
http://pixinsight.com/

Offline Harry page

  • PTeam Member
  • PixInsight Jedi Knight
  • *****
  • Posts: 1457
    • View Profile
    • http://www.harrysastroshed.com
Re: Calibration and align batch operations
« Reply #320 on: 2012 April 27 10:38:10 »
Hi

Sorry " Give it a push "  in English  slang Means  " Advertise it " ;)

Harry
Harry Page

Offline Juan Conejero

  • PTeam Member
  • PixInsight Jedi Grand Master
  • ********
  • Posts: 7096
    • View Profile
    • http://pixinsight.com/
Re: Calibration and align batch operations
« Reply #321 on: 2012 April 27 10:45:14 »
Quote
Sorry " Give it a push "  in English  slang Means  " Advertise it "

Oops!  :P  Yeah, I plan on sending a mass email announcing it. Maybe tonight! :)
Juan Conejero
PixInsight Development Team
http://pixinsight.com/

Offline Juan Conejero

  • PTeam Member
  • PixInsight Jedi Grand Master
  • ********
  • Posts: 7096
    • View Profile
    • http://pixinsight.com/
Re: Calibration and align batch operations
« Reply #322 on: 2012 April 27 10:49:33 »
Quote
** Warning: The file format reports no CFA pattern for the master dark frame, but it is being forced as per process instance parameters.

No problem at all with this. It is just a warning, not an error. It happens because the FITS format cannot tell us whether a file stores a CFA image or not. The BatchPreprocessing script 'forces' the ImageCalibration tool to load a CFA image unconditionally, hence it issues a warning because it has no way to verify if the FITS file stores a CFA image.
Juan Conejero
PixInsight Development Team
http://pixinsight.com/

Offline cs_pixinsight

  • PixInsight Addict
  • ***
  • Posts: 156
    • View Profile
Re: Calibration and align batch operations
« Reply #323 on: 2012 April 27 11:27:09 »
Quote
** Warning: The file format reports no CFA pattern for the master dark frame, but it is being forced as per process instance parameters.

No problem at all with this. It is just a warning, not an error. It happens because the FITS format cannot tell us whether a file stores a CFA image or not. The BatchPreprocessing script 'forces' the ImageCalibration tool to load a CFA image unconditionally, hence it issues a warning because it has no way to verify if the FITS file stores a CFA image.

Great, thanks for the explanation Juan.

Offline jeffweiss9

  • PixInsight Old Hand
  • ****
  • Posts: 339
    • View Profile
Re: Calibration and align batch operations
« Reply #324 on: 2012 April 27 12:19:04 »
Hi, Harry-
 If 'warm pixels' are more the problem than just 'hot' ones, how does one know how low to go with sigma in CosmeticCorrection?  In your video you suggested you had a specific quantity you dialed sigma to get. Was that just from Trial & Error for that image or is there a way to estimate it in advance?
Much thanks.
-Jeff
APM LZOS 130/780 f/6 LW CNC II APO, Riccardi 1.0 FF or 0.75 FF/FR, Tak EM-200 Temma2, FLI Microline ML-16200, Astrodon E Gen 2 filters and 5nm Ha, Orion 50mm Guider & Starlight Xpress Lodestar X2.

Offline Harry page

  • PTeam Member
  • PixInsight Jedi Knight
  • *****
  • Posts: 1457
    • View Profile
    • http://www.harrysastroshed.com
Re: Calibration and align batch operations
« Reply #325 on: 2012 April 27 12:23:31 »
Hi

Sorry it was good old trial and error , There is no magic sigma number its different for every chip/camera  ;D

I just found that if I made it more aggresiive I lost good data   and as you said before what you do not remove with Cosmetic correction I remove with
a sigma stack ( yes I dither ) :P

Harry
Harry Page

Offline mmirot

  • PixInsight Padawan
  • ****
  • Posts: 881
    • View Profile
Re: Calibration and align batch operations
« Reply #326 on: 2012 April 27 12:57:48 »
Here Juan.......................

Calibrate and Preprocessing Script V1.0
Release notes.

Designed by Kia and Juan.

The script allows for automatic sequential calbration, alignment and intitial integration of image sets. It is able to handle mutlple simultainious image sets of various combinations of binning and filters.  It does so by accessing the already robust modules contained in the PixInsight Plateform.
It elimates much of the confusion with calibrating step by step.  Master frames frames are created and can be reused.   Most of the important alignment and integration setting are avialible through the script. 

Other features include:
 * Cosmetic correction application module to remove defects such as hot/cold pixels and columns
*  direct calbration of color one shot color ccd images
*  direct calibrtion of raw files.
*  neatly sorts the output into the directory and subfolders* 
    automatic recognition of image types when used with images obtained with MaxIm Dl
 
Images parameters such as  light, bias, dark, flat, filter, and bin can be recognised automatically through their fits header when acquired through MaxIM DL.  This done the by selecting the first add images button. Images will be automatically sorted into type ( light, bias, flat) , bin and filter.

These images sets can also be easily be added with individual buttons for bias, dark, light and flat.
The add custom can be used to assign key words used in calbration for matching flats to light.
For example no key words for filters are entered in the fits by your acquistion software. No problem enter add  "Red" to your list of lights and flats and the red files will be matched up.  A similar action could be used to designate west and east flats and lights from the same pierside when using a rotator. 

An instance of script can be saved by dragging the arrow and creating an icon. This is very useful since it save all the paths and parameter for reuse.

The cosmetic correction script will be performed after calibration when checked
A saved icon has to gernerated first from the cosmetic correction module. The Icon can be reused.

Check Boxes

 bias master, dark master, flat master - check when the image loaded is a master
 CFA - check color camera
 Fits order - change for certain acquistion software.
 Rejection maps - check for when needed


What else?

Max
 





Offline georg.viehoever

  • PTeam Member
  • PixInsight Jedi Master
  • ******
  • Posts: 2132
    • View Profile
Re: Calibration and align batch operations
« Reply #327 on: 2012 April 27 13:10:24 »
Automatic typing also works by looking at name patterns (instead of FITS keyword)
Georg (6 inch Newton, unmodified Canon EOS40D+80D, unguided EQ5 mount)

Offline FunTomas

  • PixInsight Addict
  • ***
  • Posts: 135
    • View Profile
    • Astrofoto.sk
Re: Calibration and align batch operations
« Reply #328 on: 2012 April 27 13:42:10 »
...My maximum priorities now are PI Core version 1.8 and PCL version 2.0. I am working like crazy on these. PI 1.8 has to be released and fully stabilized before Summer...
Thank you Juan for your hard work. What can we look forward to? Could you tell us a little bit?

Offline Alejandro Tombolini

  • PTeam Member
  • PixInsight Jedi
  • *****
  • Posts: 1267
    • View Profile
    • Próxima Sur
Re: Calibration and align batch operations
« Reply #329 on: 2012 April 27 14:15:14 »
I am dithering, so I think, I don't need Darks. (DSLR EOS 450Da). At least, I try at the moment. There had been great discussions about that topic.

If you have an EOS 450 you definitively need darks.

Saludos.
Alejandro.