Fixed pattern noise?

[silentrunning]

I've just encountered an annoying but somewhat interesting (in that I can't explain it) artefact on last nights images.

The left image below shows 10 stacked calibrated frames. You can see a vertical banding effect that I have not seen in any of my previous images. At first I thought this might be vertical banding in the master bias files but I have used these continuously without issues. Also to check this was not the case I aligned and stacked the uncalibrated raw images and the vertical banding remained so I figured it was either present in the raw data or created by the star alignment process. To add to the mystery I could not see the artefact in any of the individual raw frames.

I began to think maybe the camera was picking up noise from a power cable or something similar and began to think what had changed since my last imaging session. The only thing I changed was the camera rotation in order to get the desired framing of the object. By chance this was exactly square with the RA/DEC grid on the sky. Then I began to think as I only dither in RA could this pattern be a result of dithering occuring along fixed columns of camera pixels so that any fixed pattern noise perpendicular to this direction would not be ironed out by the dithering. The second image shows the calibrated images stacked without registration to show how the dithering has aligned with one axis of the camera. Can anyone confirm that my logic is sound and this is a possible cause for banding? In future i'll dither in DEC as well - something I have avoided due to large backlash issues.

[STEVE333]

"Silent" - Your logic makes sense to me. However, it is odd that you aren't able to see the banding in any of the single images.

I also have bad Dec backlash, but, have settled on using both RA and Dec dithering because it makes the data so much better.

I use PHD2 and use the Lowpass algorithm on the Dec guiding. I also allow 30 sec for settling in the BYEOS dithering section (I use BYEOS to control my Canon T3). Sometimes the Dec guiding jumps and takes a while to settle (which will usually ruin that frame of data), but, the improvement in the remaining data has been worth it for me.

I have a new CEM40 EC on order and, hopefully, the backlash problems will be a thing of the past.

Best of success with resolving this banding issue.

Steve

[silentrunning]

Thanks Steve

On very close inspection it is there in the individual frames, very close to the noise level but creeps up as more frames are integrated. So I figured it most be in the bias. Image on the left below shows the master bias and the one on the right after I have applied MMT to 8 layers and discarded the residual layer which shows the bands quite well (ignore the defect pixel lines). I guess sliding up and down these bands and adding the images together (dithering along that axis and integrating) accentuates these bands as previously deduced.

Apart from this oddity dithering has always worked perfectly for me. I guess in the past my RA direction has never been plum along that camera axis before. Anyways I have pretty much sorted the backlash issue in my DEC gearing so it's time to include DEC dithering as well. 

The question now is can anyone suggest away of removing the banding as, typically, the underlying data is not that bad for once!

John

[STEVE333]

John -

You might try the  CanonBandingReduction script. I believe it wants the "bands" horizontal but not sure about that. You can try it w/wo rotating the image 90 degrees.

I've only used it a couple of times, but, my banding wasn't as severe as yours. One time it did a nice job.

Just a thought

Steve

[vicent_peris]

Hi,

CanonBandingReduction can do a good job on this image. But I just wrote a script that will completely eliminate this banding. I will release the script in the next weeks.


Best regards,
Vicent.

[STEVE333]

Fantastic Vicent.

Thanks for doing/sharing this with us. You have given a  lot to the PI community.

Much appreciated.

Steve

[Alejandro Tombolini]

Hi silentrunning,

I used to have that problem taken pictures with DSLR at high temperature and mitigate the problem using Fourier.
While Vicent's script is ready you can open FourierTransform tool and apply on the image. It will generate two more images (DFT_magnitude and DFT_phase)
Open CloneStamp tool and on DFT_magnitude delete two vertical lines that you will see over the horizontal one. You has to see it as the attached image.  Once corrected apply CloneStamp and close it.
Open InverseFourierTransform and choose as First DFT componet: DFT_magnitude and as Second DFT component: DFT_phase. Apply Globaly.

Hope this helps.

Saludos, Alejandro.   

[STEVE333]

Very clever Alejandro -

I made a copy of John's image and applied your approach. Works like a charm! Not a vertical band in sight!!!

Steve

[silentrunning]

Thanks Guys

Vicent - I look forward to your script being released.

Alejandro, very interesting approach using the FT. I had a good play around whilst in "k-space" - I'm an MRI radiographer - adding individual bright spots and creating some very interesting banding at different angles, intensities and frequencies. I find the vertical banding in the original image is caused by the bright horizontal line in the FT.

I can't however reproduce the level of yours' and Steves' results. I find as I try to clone out the lines near the centre of the image I get banding effects that are worse than the original - lower frequency banding but with more contrast - no matter how careful I am to ensure cloning of neighbouring pixels. The best I can do is to leave the middle bits of the lines which leaves banding though much less obvious than the original and at a level that can disappear after judicial stretching. Image shows original, FT, Edited FT and Inverse FT.

I wonder if it's because the image I uploaded was a low resolution JPG screen shot and I'm working with the original fits file. Maybe I could upload it somewhere to see how you guys get on with it?

Indeed - cropped my original screen grab  to the image only and applied the technique - second image below. Strangely this time the brightest line is vertical and no horizontal line, completely different from the FT of  the FITS file. The two vertical lines that Alejandra mentions are there as well which confused me a bit as they don't appear in the FT of the raw image. As with Steve, the banding is all but completely gone when the two vertical splodges are clone stamped out. Much easier process on the jpg for some reason but it would nice to use the original data.

[vicent_peris]

Hi,

Thanks, I just left a message in its corresponding board. Hope it will work for you!

https://pixinsight.com/forum/index.php?topic=13595.0


Best regards,
Vicent.

[silentrunning]

Hi Vicent

I've got a break from work at the end of week so I'm looking forward to digging into your scripts. This will be the first time I have looked at PIs scripting ability but I was a s/w engineer for 15 years so It's an exciting prospect.

Just to get a feel for things I extracted the scripts and tentatively loaded the lineardefectdetection script into the editor and set the output directory. Just to get used to compiling and executing I ran it on my image - I know it's not what's it intended for but interesting first use of the environment, I'm looking forward to running it on an integrated non-aligned set of images to generate a defect map at some point.

Well anyways the output was interesting. The line model looked somewhat like the vertical banding in my original post so I couldn't resist subtracting it from my original image and hey presto big improvement!

[vicent_peris]

Hi,

If you want to apply that pattern to the target image, write this formula in PixelMath: $T - line_pattern + med( line_pattern ) .

If forgot to say that these scripts should be applied to the linear image; any delinearization will deliver non-optimal results.

In your image, you don't need to detect any defective line since you want to simply remove the banding over the entire image, which takes always entire columns.

Best regards,
Vicent.

[silentrunning]

Thanks Vicent

Image below is the same processing done on the original linear image with your formula for subtraction.

My image has had my own defect map and clean up routines applied to it but I'm looking forward to detecting extra lines by using your routines as intended.

Really excited by this and looking into your scripts.

John

[vicent_peris]

Could you please upload the image? It's a good test.

Thanks,
Vicent.

[silentrunning]

Sure.

How do I upload the original FITs file as it's bigger than 512KB?

John