Author Topic: Calibration and align batch operations  (Read 135688 times)

Offline mmirot

  • PixInsight Padawan
  • ****
  • Posts: 881
    • View Profile
Re: Calibration and align batch operations
« Reply #150 on: 2012 March 20 21:33:10 »
custom .... ?. How does this work?


max

Offline kwiechen

  • PixInsight Addict
  • ***
  • Posts: 186
    • View Profile
Re: Calibration and align batch operations
« Reply #151 on: 2012 March 21 01:14:31 »
Hi Juan,

New version 0.3 attached:
- Regression fix: FITS headers are now correctly written to generated master calibration frames.
Stay tuned! :)

Thank you very much for this fix. I will test the new version with another set of fresh data this evening. v02 is much more performant when integrating calibration frames. The integration into the PI framework is very interesting for me - do you have some documentation for the PJSR framework?

Best

Kai


Offline mmirot

  • PixInsight Padawan
  • ****
  • Posts: 881
    • View Profile
Re: Calibration and align batch operations
« Reply #152 on: 2012 March 21 06:31:02 »
V3 works well.

It would be helpful to keep the image list and settings the next time the script is run.
Please add button to deselect individual files too.

Still trying to figure out custom?

Max

ruediger

  • Guest
Re: Calibration and align batch operations
« Reply #153 on: 2012 March 21 09:36:36 »
New version 0.3 attached:
The following applies to DSLR workflow:

If you only want to calibrate bias files and don't check "calibrate only" on the lights tab, script terminates with error:
BatchPreprocessing/BatchPreprocessing-engine.js, line 1094: Error: File I/O Error: Invalid or empty file name
The "calibrate only" should be visible always and not only on the lights tab.

Forgetting to set output directory:
BatchPreprocessing-engine.js, line 1066: Error: File I/O Error: Invalid or empty file name

Both errors close the dialog, so all other settings are lost, which is annoying.

When debayering light frames, calibrated lights are also stored on disk in RGB format, overriding my prefererred CFA monochrome setting from Format Explorer, three times larger with no advantage compared to monochrome.

Speed up suggestion: instead of "for all lights { read; calibrate; write }; for all calibrated_lights { read; debayer; write }" shorter:
for all lights { read; calibrate; debayer; write }", thus eliminating unnecessary temp files and disk i/o. Does it matter? Well, yes... calibrated 5D II files are >170MB each in RGB :)

Already mentioned, but important and please don't forget:
Allow RAW directly on each tab, so no need to do BatchFormatConversion first


Keep up the good work!

RĂ¼diger

Offline Harry page

  • PTeam Member
  • PixInsight Jedi Knight
  • *****
  • Posts: 1457
    • View Profile
    • http://www.harrysastroshed.com
Re: Calibration and align batch operations
« Reply #154 on: 2012 March 21 11:56:15 »
Hi

Still can not select the same file for Bias and Dark  :sad:

Harry
Harry Page

Offline georg.viehoever

  • PTeam Member
  • PixInsight Jedi Master
  • ******
  • Posts: 2132
    • View Profile
Re: Calibration and align batch operations
« Reply #155 on: 2012 March 21 12:10:29 »
Feedback for version 2 (did not try version 3 yet):
- I saw a problem when I wanted to add light frames. Got output like "0 of 52 frames added". Second try worked fine. Could not reproduce this.

Wishlist (probably luxary, but nevertheless nice)
- is it possible to add make like functionality (i.e. if the calibrated and registered light already exists, and nothing else changed, dont repeat the process for this image)
- tabular output, in particular for registration. Might help to identify outliers.
- tabular output of FWHM for all lights. Important to filter out bad images.
- Would be nice if it could open an Image Integration process as a last step, with basic entries alfready done.

Georg
Georg (6 inch Newton, unmodified Canon EOS40D+80D, unguided EQ5 mount)

Offline Fco. Bosch

  • Member
  • *
  • Posts: 66
    • View Profile
Re: Calibration and align batch operations
« Reply #156 on: 2012 March 21 12:32:21 »
Would be possible a general option for "create masters" or so? (integration of bias, dark and flat, without the light frames to calibrate) Now the metod  seems a bit confuse   :surprised:


Fco. Bosch

Offline mmirot

  • PixInsight Padawan
  • ****
  • Posts: 881
    • View Profile
Re: Calibration and align batch operations
« Reply #157 on: 2012 March 21 13:57:07 »
Feedback for version 2 (did not try version 3 yet):
- I saw a problem when I wanted to add light frames. Got output like "0 of 52 frames added". Second try worked fine. Could not reproduce this.

Wishlist (probably luxary, but nevertheless nice)
- is it possible to add make like functionality (i.e. if the calibrated and registered light already exists, and nothing else changed, dont repeat the process for this image)
- tabular output, in particular for registration. Might help to identify outliers.
- tabular output of FWHM for all lights. Important to filter out bad images.
- Would be nice if it could open an Image Integration process as a last step, with basic entries alfready done.

Georg

I think the Blink Module is the place to add a FWHM function.
Images need to be fully reviewed and screened first.
We risk adding to much complexity to the script.

Don't get me wrong Georg. It would great to have this as part of PI. I think the blink module is the logical step to look, analyse and sort images.

I think Juan plans to add integration.

Max




Offline kwiechen

  • PixInsight Addict
  • ***
  • Posts: 186
    • View Profile
Re: Calibration and align batch operations
« Reply #158 on: 2012 March 21 14:13:00 »
Would be possible a general option for "create masters" or so? (integration of bias, dark and flat, without the light frames to calibrate) Now the metod  seems a bit confuse   :surprised:

create masters should work, if the light tab is left blank. I am using this to generate masters for reuse.

Feedback for version 2 (did not try version 3 yet):
- I saw a problem when I wanted to add light frames. Got output like "0 of 52 frames added". Second try worked fine. Could not reproduce this.

Wishlist (probably luxary, but nevertheless nice)
- is it possible to add make like functionality (i.e. if the calibrated and registered light already exists, and nothing else changed, dont repeat the process for this image)
- tabular output, in particular for registration. Might help to identify outliers.
- tabular output of FWHM for all lights. Important to filter out bad images.
- Would be nice if it could open an Image Integration process as a last step, with basic entries alfready done.

Georg


Personally I think the script should do calibration and alignment only because this could be done with a set of parameters preset. Checking the calibrated lights should be done with another tool. The script dialogs are somewhat limited in interactivity due to their modal behavior.

Best,

Kai


Offline Harry page

  • PTeam Member
  • PixInsight Jedi Knight
  • *****
  • Posts: 1457
    • View Profile
    • http://www.harrysastroshed.com
Re: Calibration and align batch operations
« Reply #159 on: 2012 March 21 15:15:53 »
Hi

I have just done a cal run on some Red subs all ok I then deleted the red flats and red lights and selected the green flats and green lights and the script fell over
say could not intergrate less than three subs ( flats ) !!!!!!!!

Happens every time

Harry
Harry Page

Offline Harry page

  • PTeam Member
  • PixInsight Jedi Knight
  • *****
  • Posts: 1457
    • View Profile
    • http://www.harrysastroshed.com
Re: Calibration and align batch operations
« Reply #160 on: 2012 March 21 15:38:58 »
Hi

Ok  after the first run the use master ( flat in this case) box gets automaticaly checked to use the newley created master flat , if you do not uncheck this box first
before loading up new Flatsfor the second run  the script does not like it  :yell:

Harry
Harry Page

Offline Juan Conejero

  • PTeam Member
  • PixInsight Jedi Grand Master
  • ********
  • Posts: 7089
    • View Profile
    • http://pixinsight.com/
Re: Calibration and align batch operations
« Reply #161 on: 2012 March 21 15:46:49 »
Quote
Personally I think the script should do calibration and alignment only

I completely agree. Integration of light frames requires trial-and-error and analysis work to achieve optimal results. It is a delicate task that cannot be scripted on a general basis, especially given the complexity of our ImageIntegration tool (which will grow in a near future)..

Quote
- is it possible to add make like functionality

This would be nice.

Quote
- tabular output, in particular for registration. Might help to identify outliers.

Also agreed.

Quote
- tabular output of FWHM for all lights.

As Max has noted, this should be part of a different tool (probably implemented as a C++ module, not as a script for performance reasons). I guess Nikolay is already working on such tool. This task should be performed to filter out bad images before calibration and alignment.

Thanks for the ideas!

To everybody: Please bear in mind that this script is in its early development stages---it is still version 0.3 after all!

For monochrome CCD and color OSC images the script is fully usable right now (besides particular problems and special requirements), and provides nice results. However, it is definitely still too crude for DSLR, although it can be also used without problems if bias and dark frames are converted to FITS before the script. We have decided to push the script hard to achieve a first release version as soon as possible.
Juan Conejero
PixInsight Development Team
http://pixinsight.com/

Offline Juan Conejero

  • PTeam Member
  • PixInsight Jedi Grand Master
  • ********
  • Posts: 7089
    • View Profile
    • http://pixinsight.com/
Re: Calibration and align batch operations
« Reply #162 on: 2012 March 21 15:56:36 »
Replying to myself:

Quote
This task should be performed to filter out bad images before calibration and alignment.

Actually, after alignment to facilitate finding of stars on the images.
Juan Conejero
PixInsight Development Team
http://pixinsight.com/

Offline mmirot

  • PixInsight Padawan
  • ****
  • Posts: 881
    • View Profile
Re: Calibration and align batch operations
« Reply #163 on: 2012 March 21 16:06:23 »
No script for integration...I disagree :'(

Iteration should not be reason to avoid and an automated approach.

Besides I hoping you can find  a way to iterate with this script based on the aligned files.

Are the images written to a cache?
Can a ROI be implemented on a initial integration?

Max

Offline mmirot

  • PixInsight Padawan
  • ****
  • Posts: 881
    • View Profile
Re: Calibration and align batch operations
« Reply #164 on: 2012 March 21 16:10:51 »
What does custom do?

Max