Issue when master flat is more than 13 files

[Philippe B.]

Dear all
Something strange...


When I compute a master flat with up to 13 files, the result is a max value around 0.74
When I add 1 or 10 files more, result drop to 0.46


You can download here the archive with 15 files, the 2 master resulting, the process icon.


http://astroccd.eu/transfert/flat.zip


I didn't calibrate flat but result is the same with calibrated files.

It happens with last PI release and OSX or Windows version (I cannot test Linux right now).

thanks
Philippe

[oldwexi]

Hi Philippe!
Great test data, everything is there! Very seldom here...

I do not know the reason why you have reduced the value of
Percentile low:  to    0.010     ?

Setting the Percentil low a littler higher for examle to 0.030 gets you the results
you want even with 15 images.

I dont know the interna of "Percentile Clipping" +  "Percentile low" algorithm. Maybe Juan can answer this.

But so far you have a solution...

Gerald

[Philippe B.]

thank you Gerald but 0.030 or more doesn't solve the problem, even any other settings (median, average, or sigma clip... )


We use percentile clipping very low because we are doing our flats on the sky, early the morning. So there can be some bright stars in the image.


cheers

[Juan Conejero]

Hi Philippe,

Disable output normalization (select No normalization in the Image Integration section) and your integrated flat will always have a maximum value equal to the average maximum in the input set.

However, the average value of a valid master flat frame should be irrelevant. The ImageCalibration tool will always normalize the master flat, so the resulting maximum and minimum values in the integrated flat are not important; what is important is the shape of the two-dimensional surface represented by the flat, and it will be the same irrespective of whether you normalize it or not during integration.

Besides that, with 15 frames, why are you using percentile clipping? Sigma clipping with or without Winsorization should work better.

[Juan Conejero]

Hi again Philippe,

These frames have a row of saturated pixels. The first row of each flat frame has a constant value of 65535. This is obviously wrong and is, for some reason that I have to investigate further, affecting ImageIntegration's normalization routines in this particular case.

To fix this problem with these flat frames, you can use PixelMath with an ImageContainer. This is the PixelMath expression I've used:

iif( y() == 0 && $T == 1, pixel( $T, x(), y()+1 ), $T )

This expression will simply copy the second row of pixels to replace the first row. With this change, you can integrate your 15 flat frames with any rejection settings without problems. The maximum value of the integrated master flat will be about 0.78, and the mean value about 0.69, as expected.

[Philippe B.]

Hi JuanMany thanks for your answer.It is working fine when selecting no normalization.

In the past, we use Percentile Clipping because of flats taken on the sky in the morning, so it stays some bright stars. This was an idea from PI team at the beginning