Need help eliminating hot pixels - ASI2600MC

joelshort

Well-known member
I have a new camera (ZWO ASI2600MC) and I am having a hard time eliminating some hot pixel streaks in the resulting calibrated/stacked image. Despite using matching master dark, bias and flat frames, and using CosmeticCorrection I still get lots of streaks in the stacked image, as seen in the attached image.

My workflow is
Calibrate Lights with master Bias/Dark/Flat
CosmeticCorrection
Debayer
StarAlignment
ImageIntegration (LinearFit clipping)

The attached image is what I get.

Some additional notes:
-I have stacked the images without using CosmeticCorrection and the result is only slightly worse that using CC.
-I have tried adjusting the CC hot pixel count anywhere from 3000 to 150,000. The image appears somewhat better at around 60,000 but anything beyond that gives me strange results in the stacked image.
-I also have an ASI6200MM that uses an identical sensor (just bigger). If I don't use CC or rejection stacking then the resulting integrated image has identical looking hot pixel streaks. However, normal processing (my typical workflow above) results in the hot pixels being almost completely eliminated.
-I have tried using flatdarks to calibrate the flats instead of bias. Same result.

I really have no idea why these hot pixels are not being eliminated with CC, so any advice would be much appreciated.
 

Attachments

  • post-37568-0-35510100-1586191318_thumb.jpg
    post-37568-0-35510100-1586191318_thumb.jpg
    53.4 KB · Views: 207
Hi,

Looking at your image I don't think those are hot-pixels. Hot-pixels should be just a small dot but those things seem to have some "length/movement". Two suggestions: have a closer look at your flats and perhaps try re-stacking without the Bias.

space is not black
John
 
When you take your images do you dither the frames? I have had similar noise on my masters that disappeared when I started to do this.
 
Looks like "walking noise", caused by not correctly applied image clibration in combination with a slight drift during the whole capturing session. It has nothing to do with the calibration of the flat frames. Please describe how you prepared the MasterDark and disclose you complete calibration settings (best as a screen section showing the ImageCalibration window).

Bernd
 
Looking at your image I don't think those are hot-pixels. Hot-pixels should be just a small dot but those things seem to have some "length/movement". Two suggestions: have a closer look at your flats and perhaps try re-stacking without the Bias.
This is part of a stacked, calibrated image after CosmeticCorrection and stacking rejection. That's why the hot pixels appear to trail.

When you take your images do you dither the frames? I have had similar noise on my masters that disappeared when I started to do this.
I did not dither the frames. Because I have a tandem scope/dual camera setup I cannot dither one camera without messing up the image in the other camera. But I've been doing this for two years (with different cameras) without issue.

Looks like "walking noise", caused by not correctly applied image clibration in combination with a slight drift during the whole capturing session. It has nothing to do with the calibration of the flat frames. Please describe how you prepared the MasterDark and disclose you complete calibration settings (best as a screen section showing the ImageCalibration window).
I took 30 darks at the same exposure, gain, offset and temperature as the lights (300s, 0, 50, -10C).
MasterDarkStacking.png


Here are my ImageCalibration settings:
ImageCalibration.png
 
Hi Joel,

your settings for the calibration of the light frames are not wrong, but somewhat cumbersome.

If you want to upload a light, flat, dark and bias frame in FITS format, and the MasterFlat, MasterDark and Master Bias in XISF format I will take a look.

Bernd
 
your settings for the calibration of the light frames are not wrong, but somewhat cumbersome.
As mentioned in the OP, I also have an ASI6200MM (mono) and these same settings, combined with CosmeticCorrection and rejection stacking, do eliminate the hot pixels.

If you want to upload a light, flat, dark and bias frame in FITS format, and the MasterFlat, MasterDark and Master Bias in XISF format I will take a look.
Thanks very much Bernd.
Unfortunately I do not have the original individual flat frames. I have included a single calibrated flat frame (bias subtracted) and the master flat frame.

Download the files here.
 
This is strange. Your integration shows that there are pixels with higher signal in the light frames that are not noticeable in the MasterDark. It seems as if the MasterDark is not matching, despite same gain, sensor temperature and exposure time. Though applying the MasterDark improves the frame, its effect is insufficient.

First of all I would capture new dark frames, make a new MasterDark and compare old and new MD and the integration results with old and new MD after calibration, debayering and registering.

---

I also tried CosmeticCorrection, and it failed completely when I applied it to the calibrated light frame in CFA format (of course, 'CFA' was enabled in the 'Output section').

Then I made an interesting observation: I split the calibrated light frame into the 4 CFA channels. I applied CosmeticCorrection (CFA NOT enabled in the 'Output' section) with 'Use Auto detect', Hot Sigma 7.5 to each of the CFA channel. Subsequently I merged the CFA channels and debayered. The result was a great improvement! Perhaps you can try this and report whether the final integration is OK.

This approach is somwhat tedious, but it worked. I don't know why CC didn't work on the CFA data. Perhaps this is a topic for Juan?

Bernd
 
First of all I would capture new dark frames, make a new MasterDark and compare old and new MD and the integration results with old and new MD after calibration, debayering and registering.
Yesterday as an experiment I captured 10 new dark frames (I normally capture 30) and created a new MasterDark from those 10. Same (negative) result with that new MasterDark.

I'll try splitting the channels as you suggest and see what happens. Just to be clear, you did NOT use the MasterDark for CosmetiCorrection? I'm guessing because of the bayer matrix in the MasterDark vs mono channels?
 
Just to be clear, you did NOT use the MasterDark for CosmetiCorrection? I'm guessing because of the bayer matrix in the MasterDark vs mono channels?
Yes, I did not. The MasterDark has done its job. Different pixels have remained that cannot be corrected with the MasterDark.

Bernd
 

Attachments

  • CC_Settings.JPG
    CC_Settings.JPG
    58.8 KB · Views: 75
  • Before.JPG
    Before.JPG
    182.5 KB · Views: 74
  • After.JPG
    After.JPG
    180.9 KB · Views: 114
First of all I would capture new dark frames, make a new MasterDark and compare old and new MD and the integration results with old and new MD after calibration, debayering and registering.
What are your settings for integrating the MasterDark? Perhaps I am not integrating correctly.
 
Is there a way to batch split the channels? I have 63 images...
EDIT: I found the BatchChannelExtraction script. But what "4 CFA" channels? Aren't there only 3? RGB?
 
almost every CFA sensor has 4 pixels per bayer matrix - R, G, G and B. when you debayer the 2 green channels get averaged together which is why you never see the 2 green channels. but it does account for why CFA images seem to have a green cast after debayering - the green SNR is 2x that of the other channels and that fools STF when the channels are locked.
 
What are your settings for integrating the MasterDark? Perhaps I am not integrating correctly.
Cited from my guide ( https://pixinsight.com/forum/index.php?threads/for-beginners-guide-to-pis-imagecalibration.11547/ ):

Section 'Image Integration': Combination 'Average', Normalization 'No normalization', Weights 'Don't care (all weights = 1)'
Section 'Pixel Rejection (1)': Normalization 'No normalization'
Which rejection algorithm in section 'Pixel Rejection (1)' is appropriate depends on the number of used frames. See the PixInsight Reference Documentation "ImageIntegration" [3] for recommendations. With the settings in section 'Pixel Rejection (2)', a fine tuning is achievable in order that only unwanted signal (e.g. caused by cosmic ray artifacts) is rejected.

If I remember correcctly (I don't have your ), you used the correct settings for dark frame integration.

Bernd
 
almost every CFA sensor has 4 pixels per bayer matrix - R, G, G and B. when you debayer the 2 green channels get averaged together which is why you never see the 2 green channels. but it does account for why CFA images seem to have a green cast after debayering - the green SNR is 2x that of the other channels and that fools STF when the channels are locked.
Right, but Bernd made it sound like he split the light frame into 4 channels. When you debayer and split the channels you get 3, RGB. And BatchChannelExtraction only gives 3 channel options for each type.

And my next question...how do I batch recombine all those separate channels? That would be quite cumbersome if I have to do it all manually.
 
Is there a way to batch split the channels? I have 63 images...
EDIT: I found the BatchChannelExtraction script. But what "4 CFA" channels? Aren't there only 3? RGB?
I split the CFA data (no debayering involved in this case!) with the SplitCFA process to produce the monochrome images CFA0, CFA1, CFA2 and CFA3 with the halved width and halved height. Then I applied CosmeticCorrection to each of these monochrome images. Subsequently I merged (with MergeCFA) the cosmetically corrected monochrome images to CFA data (with the original resolution) again. This file was finally debayered.

As far as I know there is no Batch Script for these operations, neither for SplitCFA nor MergeCFA.

Bernd
 
in theory you can probably use SplitCFA with ImageContainer but MergeCFA probably does work with ImageContainer since there are multiple input files.

if you are OK with superpixel debayering you can just register and integrate all the images that came out of SplitCFA - no need to recombine them back to CFA. i guess you could recover your original file dimensions using Drizzle integration, which might actually give a superior result. but BayerDrizzle is off the table and that might give you an even better result.
 
I re-took a fresh set of darks. Same result. :mad:

SplitCFA works great for all the images at once. It's recombining them that is the problem. That will take forever. I just don't understand why CosmeticCorrection doesn't work on the raw image.

if you are OK with superpixel debayering you can just register and integrate all the images that came out of SplitCFA - no need to recombine them back to CFA. i guess you could recover your original file dimensions using Drizzle integration, which might actually give a superior result.
Could you say more about how to do this? Will I end up with images that are the original size before SplitCFA?
After 18 years this is my first OSC camera and it feels like learning everything all over again.
 
the flow would be the same as with a mono camera. if you have N subs, then you'll have N R subs, N*2 G subs and N B subs as though taken with a mono camera.

when you do StarAlignment, there's an option to generate Drizzle data. so you check that. then when you run ImageIntegration, you need to load those drizzle files and also tick "generate drizzle data". when ImageIntegration is done (tuned however you want it to be) you get the normal image like always. but at that point you open the Drizzle process and load all the drizzle data that was just updated by ImageIntegration and run. i think the default drizzle setting is 2x drizzle, so yes, you end up with master lights that have the same dimensions as the original files. the bonus here is that no interpolation was ever done, either while debayering or while registering so the drizzled master will be of higher quality.

rob
 
the flow would be the same as with a mono camera. if you have N subs, then you'll have N R subs, N*2 G subs and N B subs as though taken with a mono camera.
When using SplitCFA 4 files are generated (CFA0, CFA1, CFA2, CFA3). How do you know which of those correspond to RGB? If the bayer pattern is RGGB, does CFA0=R, CFA1-2=G and CFA3=B?
 
Back
Top