Author Topic: bug in Integration module  (Read 3753 times)

Offline mmirot

  • PixInsight Padawan
  • ****
  • Posts: 881
bug in Integration module
« on: 2009 May 24 17:18:44 »

 1 My system crashes each time when I try " averaged sigma clipping"

 2 The prior path is not saved after you select images. You are back to the root directory



 3 ( not a bug)
If you are using 32 floating point FIT PI sends the dialogue box asking about rescaling each time.
Can we get an say yes to all rather than hitting Ok for 35 images?

 4  I tried using a 32 integer output from CCDStack but they all appear a flat gray with no real image data. 
I have never tried 32 integer from Stack before,    so this may a known issue with it's output.
 
 The default sigma clipping did the best job rejecting my dark column defects of any program. Great output!  :)

Max


Offline Juan Conejero

  • PTeam Member
  • PixInsight Jedi Grand Master
  • ********
  • Posts: 7111
    • http://pixinsight.com/
Re: bug in Integration module
« Reply #1 on: 2009 May 25 00:49:37 »
Hi Max,

Quote
1 My system crashes each time when I try " averaged sigma clipping"

Not actually a bug. It is Microsoft's C++ compiler which seems unable to compile the ImageIntegration module correctly. Refer to this post:

http://pixinsight.com/forum/index.php?topic=1158.msg5714#msg5714

If you have a Linux or Mac OS X box at hand, average sigma clipping works nicely. The error happens with this rejection algorithm exclusively. Today I'll request a trial license for Intel's C++ compiler for Windows, to see if I can fix this issue (and a few other compilation problems ith M$'s compiler).

Quote
2 The prior path is not saved after you select images. You are back to the root directory

Yes, this is a known regression in PI 1.5 for Windows. Multiplatform file dialogs (Edit > Global Preferences > File I/O Settings > Use native file dialogs (uncheck)) work perfectly though.

Quote
If you are using 32 floating point FIT PI sends the dialogue box asking about rescaling each time.
Can we get an say yes to all rather than hitting Ok for 35 images?

Of course. Open the Format Explorer window, and double click on the FITS item (left column). On the FITS Format Preferences dialog, Floating Point Out Of Range Policy section, select the "Normalize to the default input range" option for "How to proceed".

Quote
I tried using a 32 integer output from CCDStack but they all appear a flat gray with no real image data.
I have never tried 32 integer from Stack before,  so this may a known issue with it's output.

Yes, this is certainly a known issue. The data are there, and you can use them in PixInsight without problems. Open the Process Explorer window, expand the IntensityTransformations category, then open the Rescale process. Apply Rescale with default parameters to one of these images, then use STF as usual, and voilĂ , everything's there.

The problem is actually quite simple. These are 32-bit integer FITS images. As you know, one of these images can store 232 discrete values (4,294,967,296 different values). However, these files store less than 216=65,536 discrete values (at least this happens in all CCDStack 32-bit integer images that I've seen so far).

But the problem isn't in this fact, actually. The real problem is that the 65536 values have been stored into a contiguous region of the available dynamic range, centered around 232/2 (the middle point of the histogram). Consequently, we have an extremely sharp peak (virtually a Dirac delta) that concentrates less than 65536 values in the middle of a hugely vast numerical range of 4294967296 values. In more understandable figures, this means that only a fraction of 0.000015 of the available dynamic range is being used by these images. Note that 16-bit histograms (which are being used internally by PixInsight) are unable to manage that infinitesimal ratio of dynamic range usage (we would need at least a 24-bit histogram to provide reasonable resolution for these images, but of course we don't want to waste 32 MB of RAM to represent the histogram of a 16-bit image).

I won't comment about what I think about the way these files have been generated. Everybody is free to work as he/she thinks is better, and of course everybody is free to make things more or less difficult to share data with other applications. In my opinion, you'd better store these images as 16-bit FITS files, which is what they are. You'd avoid a lot of problems in that way.

Quote
The default sigma clipping did the best job rejecting my dark column defects of any program. Great output!

(Edited: I forgot to answer this). Thank you! Yes, sigma clipping is indeed the best option (IMO) for sets of more than 8 or 10 images. For smaller sets, my preference if percentile clipping. However, beware that there's a bug in PI 1.5 that can cause incorrect rejection of small sets of an even number of images:

http://pixinsight.com/forum/index.php?topic=1158.msg5599#msg5599

This doesn't affect you if you're integrating many images, and of course there's no problem with 35 images (odd number). This will be fixed in the upcoming PI 1.5.2.
Juan Conejero
PixInsight Development Team
http://pixinsight.com/

Offline mmirot

  • PixInsight Padawan
  • ****
  • Posts: 881
Re: bug in Integration module
« Reply #2 on: 2009 May 25 10:52:38 »
Boy that was great response Juan.  Thanks

Too bad I did not find anything new.
That is no help to you. :-[



"This doesn't affect you if you're integrating many images, and of course there's no problem with 35 images (odd number). This will be fixed in the upcoming PI 1.5.2."

I lied. It was 26 images. It worked fine.
I generally end up with a minimum 15 images take 400 second exposures.
 If I do narrow band then it is less since I take 1200-1800 second exposures using a off axis guider. 

Max


Offline mmirot

  • PixInsight Padawan
  • ****
  • Posts: 881
Re: bug in Integration module
« Reply #3 on: 2009 May 25 11:11:42 »
In my opinion, you'd better store these images as 16-bit FITS files, which is what they are. You'd avoid a lot of problems in that way.


Your so right. I forget these are fresh calibrated images and thus should start life in their native Fits formats.
No truncation problems are possible since there are born as 16 bit integers.

Max