Need help eliminating hot pixels - ASI2600MC

In CFA data, the assignment of the CFA channels is like this when the camera has the bayer pattern 'RGGB':

Code: 
0  2      R  G
1  3      G  B

CFA data are not RGB images! The RGB image has to be interpolated (debayering) or CFA drizzle has to be applied. I don't know whether Rob's idea will work without dithering between the frames, but it's worth a try.

Bernd
 
OK so I went through the process that Rob outlined and the results are much better as Bernd indicated as well. So thanks for helping me through this.

However, it still puzzles me that darks don't do a better job at calibration, and even more so that CosmeticCorrection doesn't take care of the leftover hot pixels either. I have had no issues with other cameras (mono) using this same workflow that I've been using for years.

And another mystery...since the bayer matrix results in a stronger green signal, I would expect there to be a green cast to images that are debayered. This is true for an unprocessed sub straight from the camera. But as soon as I calibrate the image with a MasterBias, MasterDark, MasterFlat the resulting image is blue by a factor of about 2x. EDIT: I've discovered that it is when I calibrate with my MasterFlat that the image results in blue instead of green. I'm not sure what, if anything, this means.
 
Take a look at the histogram of your MasterFlat: the CFA3 channel (= blue) is by far the weakest. During flat field correction, the intermediate image is divided pixel by pixel by the MasterFlat and multiplied by the mean of the MasterFlat (a constant) which is the average of ALL channels. In your case (weak blue channel) this results in a strong amplification of the blue channel. Such a color shift during flat field correction is normal. The calibrated light frames are not yet color calibrated. This has to be done after image integration (e.g. by applying PhotometricColorCalibration).

Bernd
 
Last edited:
joelshort said:
When using SplitCFA 4 files are generated (CFA0, CFA1, CFA2, CFA3). How do you know which of those correspond to RGB? If the bayer pattern is RGGB, does CFA0=R, CFA1-2=G and CFA3=B?
Click to expand...

yes, i believe that is correct. the only caveat would be for some DSLRs libRAW changed how the file dimensions are computed and although almost all DSLRs are RGGB the bayer pattern can be shifted because of this. but i don't think the ASI2600MC was affected by this problem.

you can always take a sub and superpixel debayer it and split the channels, and then compare what you see in each channel with it's corresponding CFA* files as a sanity check. of course if you took a picture of a white wall this would not help but certainly most DSOs are going to show differences between each channel. if you were really concerned you could take 3 t-shirt flats thru a red, green and blue t-shirt and do the comparison.

rob
 
bulrichl said:
In CFA data, the assignment of the CFA channels is like this when the camera has the bayer pattern 'RGGB':

Code: 
0  2      R  G
1  3      G  B

CFA data are not RGB images! The RGB image has to be interpolated (debayering) or CFA drizzle has to be applied. I don't know whether Rob's idea will work without dithering between the frames, but it's worth a try.

Bernd
Click to expand...

yes, sorry, i assumed the input files were dithered since joel has been at this a long time and is most likely dithering.

rob
 
Actually I am not dithering. The reason is because I have a tandem scope/dual camera setup and if I dither one it messes up the other.
That said the above processing workflow worked rather nicely.
 
oh you might have mentioned that. well then it's kind of a miracle that drizzle worked. maybe there is enough natural drift in the images for it to work out.
 
There's either a slight drift, or a slight flex between the scopes. I can usually get 10min subs like this, for sure 5min subs, without star trailing.
 
joelshort said:
Actually I am not dithering. The reason is because I have a tandem scope/dual camera setup and if I dither one it messes up the other.
That said the above processing workflow worked rather nicely.
Click to expand...
I am curious. Did you further tweak the sigma value for the individual channels? Can you please post an image of your result that corresponds to the image in your first post of this thread?

Bernd
 
bulrichl said:
I am curious. Did you further tweak the sigma value for the individual channels? Can you please post an image of your result that corresponds to the image in your first post of this thread?
Click to expand...
I simply did exactly as you and Rob suggested.
Calibrate Lights with MasterBias, MasterDark, MasterFlat
SplitCFA on the calibrated Lights
Run CosmeticCorrection on all the split files, (no dark frame, Automatic with hot sigma 7.5)
StarAlignment on all channels with Drizzle Files
ImageIntegration on all channels with Drizzle Files
DrizzleIntegration

Here's the result.
PostSPLITCFA.png
 
Thank you for posting this. I think this is looking fine. Is it also satisfactory for you?

I have created a bug report for the handling of CFA data by CosmeticCorrection. I hope that this issue can be fixed. Otherwise we would need a revised CFAMerge process that can be applied to files (and not only to views), in order to be used in connection with an Image Container.

Bernd
 
Yes, this result is fine to me as well. The process is just very tedious and it would be much better if CosmeticCorrection would be able to take care of it.
 
Guys, I discovered something very strange. If I DON'T use CosmeticCorrection the image comes out pretty good.

When I first saw these hot pixel streaks in the integrated image, I instead tried to use WBPP with the standard settings and I got the same bad result. A friend of mine took my data (Lights and Master calibration frames) and simply used BPP with the standard settings (Linear Fit, Low 4, High 2) and the results were good.

I was confused because I had tried WBPP (not BPP) and got bad results, but when I tried BPP as my friend did the resulting image was free of hot pixel streaks.

So I went back and did my usual calibration and stacking workflow but simply left out CosmeticCorrection and the image came out great. How is it possible that CosmeticCorrection was actually adding back in hot pixels????
NoCC.png
 
without combing thru all the data, i suppose it's possible that without CC the hot pixels were of high enough intensity that they were more easily rejected during ImageIntegration. if CC left your hot pixels in sort of a "warm" state then they would not look like outliers to ImageIntegration. but without running all the subs (CC and non-CC) this is just a guess...

what high/low sigmas are you using during integration?
 
hm, i still use windsorised clipping so i don't have a good idea whether or not 2 under linear fit would not be aggressive enough. i suppose you can visually inspect a debayered CC vs corresponding non-CC frame to get an idea of what the hot pixel intensities are.
 
It's really, really hard to see any difference between a non-CC and CC corrected frame. If anything the CC corrected sub appears to have less hot pixels, as one would expect. But when I stack those I get the hot pixel trails.
 
joelshort said:
Guys, I discovered something very strange. If I DON'T use CosmeticCorrection the image comes out pretty good.

When I first saw these hot pixel streaks in the integrated image, I instead tried to use WBPP with the standard settings and I got the same bad result. A friend of mine took my data (Lights and Master calibration frames) and simply used BPP with the standard settings (Linear Fit, Low 4, High 2) and the results were good.

I was confused because I had tried WBPP (not BPP) and got bad results, but when I tried BPP as my friend did the resulting image was free of hot pixel streaks.

So I went back and did my usual calibration and stacking workflow but simply left out CosmeticCorrection and the image came out great. How is it possible that CosmeticCorrection was actually adding back in hot pixels????
Click to expand...

Your last sentence is nonsense. CosmeticCorrection is not adding back in hot pixels.

I suspect the same as Rob and will express it more rigorously: probablythe integration that contained the streaks was performed with NO pixel rejection.

Bernd
 
bulrichl said:
Your last sentence is nonsense. CosmeticCorrection is not adding back in hot pixels.

I suspect the same as Rob and will express it more rigorously: probablythe integration that contained the streaks was performed with NO pixel rejection.

Bernd
Click to expand...
I agree that last sentence is nonsense, as it was meant to be. Obviously I know that CC can't be adding in hot pixels.

But you are also making an assumption that is not true, as I stated in the original post and as I have tested multiple times. I AM using pixel rejection during integration (Linear fit to be precise).

My workflow is
Calibrate Lights with master Bias/Dark/Flat
CosmeticCorrection
Debayer
StarAlignment
ImageIntegration (LinearFit rejection + large scale pixel rejection)

If I do the above the resulting image has those hot pixel streaks. If I do the above but simple eliminate CC I do not get the hot pixel streaks in the integrated image. I don't have an explanation for that. It makes no sense.

In any event I have moved on, since I found a solution to the problem.
 
You must log in or register to reply here.
Back
Top