Banding in master image

[Ryderscope]

I have just completed running through a data set for NGC 104 using 1.8.5 for the first time. I am using the Batch Preprocessing script to calibrate, register and integrate my data. I am following through the exact same steps as I have with all previous data sets.

My issue is that there is banding present in each of the LRGB masters produced by the script. I have never seen this before with my processing. I have attached a copy of the master Red channel and one of the subs. The banding is present in the integrated master but is not present in any of the subframes. The banding present in each of the masters is also in the exact same position and fades from left to right across the image.

For the Batch Preprocessing script I supplied bias, darks, flats and lights (approximately 60 subs per channel). Each subframe is only 30 seconds (NGC104 is a bright object). The darks that I used are 90 second darks and I selected the optimise dark frame option. I selected the best of the green subframes as my image reference.

My hardware setup is: TAK TSA120, Feather touch focuser, TSA130 focal reducer, ZWO OAG, QSI683 WS-8 camera.
Filters are Astronomik 100nm LRGB. Image capture using SGP Pro.

I will note that the darks and bias frames are dated from last December so I do need to take another set but I would not have thought that it would make this much of a difference.

Any other clues for me please as a starting point so that I can see where this problem may lie?

Edit: I have taken a new set of bias and 30 second dark frames, rerun the batch preprocess and the banding is still there.

[Ryderscope]

Further research on my problem shows that the banding is in fact present in the master bias and master dark which are output from the batch preprocessing script. See images below. I went back through several previous processing activities and the same banding is present. I suspect that I am only seeing the issue in the light frame masters with this image because of the short (30 second) sub frame exposures. Looks like I will need to go back to basics and go through the process of building master darks and master bias first. Will try this and see how it goes.

[pfile]

i think you are right - some DSLRs have this problem at particular ISOs. you could try a different ISO to see if the problem is less pronounced.

but yes 30s is pretty short and there is not enough time for real signal to build up...

rob

[Ryderscope]

I've found that the issue that I observe is actually due to banding in the master dark frames and I would like to understand what this really means. To this end I have prepared a couple of master dark frames and uploaded them to a shared drop box folder along with the process icons that I have been using. The link to the shared folder is https://www.dropbox.com/sh/hk3llbbu3eaj77r/AACyTVY5QOCLbJo_EWYfdzlSa?dl=0.

What I find is by using the recommended settings for creating master darks, the master dark has vertical bands that are concentrated on the left hand side of the image. An inspection of the sub frames does not show any evidence of these vertical bands. For the testing I was integrating 30 sub frames at 300s exposures each. I used linear fit clipping for the rejection algorithm and accepted the defaults for the low and high rejection of 5.0 and 2.5 respectively.

I found that by experimenting with the settings I can make the banding go away be lowering the linear fit high parameter to a value of .5. I have attached a copy of a screenshot of the examples of the master dark images that I produced as well as a screenshot of the ImageIntegration process window.

Included in the drop box folder are:

- The two different master dark images
- The two image process icons used for each of the test dark frames
- A sample of one of the master dark sub frames

So, my questions are:
1. Why is this banding present in the integrated frame when there is no evidence of it in any of the sub frames?
2. Does it really matter?
3. Why does lowering the "linear fit high" parameter make the banding go away?
4. If I produce a master dark by lowering the "linear fit high" parameter to .5 will it still be usable for image calibration?

It would be good if there are some ideas on this as it is puzzling me.

Many thanks in advance,
Rodney.

[msmythers]

Rodney

It might be helpful to have more then one dark sub so an independent master could be created and analyzed.



Mike

[Ryderscope]

Rodney

It might be helpful to have more then one dark sub so an independent master could be created and analyzed.

Mike

Yes, very true. I've added a sub folder called 'darks' and uploaded the 30 dark sub frames so there will be plenty to choose from. It will take some minutes for the files to complete uploading. Let me know if there are any issues with accessing the data.

[msmythers]

Rodney

I am far from an expert when it comes to image analysis but I would say the problem is in your dark subs. I did a simple test. I took the first 10 dark subs from your upload and ran them through ImageIntegration with the same settings you used except I used no pixel rejection. So this as far I I know should be just the average of the 10 subs. I got the banding. Not as bad as if there were 30 of your subs but not much different.

I then did a very simple pixel math averaging of your first 10 darks subs. (1+2+3...10)/10 and created a new image. Here is the result.

Both images have the same STF stretch applied.


Mike

[Ryderscope]

Hmmmm, most interesting Mike and thanks for going to the trouble of running those tests. At least it shows that I wasn't doing anything silly myself. A visual inspection of the darks by running them through the blink processs does not reveal anything obvious so I am at a loss to explain what is happening. I will see if I can track down some darks from other QSI683 owners to use as a comparison.

[Ryderscope]

Ok, I have found a number of threads including one here on Cloudy Nights https://www.cloudynights.com/topic/498053-qsi-683-vertical-banding-issues-solutions/ regarding vertical banding on master dark images from a QSI683 which is the same camera that I am using. It seems that this is a common issue. In my case it only seems to cause a problem when I am shooting short subs (e.g. 30s) so I can work around it. It is still strange though how it doesn't seem to be apparent in the individual dark frame subs.

[msmythers]

Rodney

Think of this like a weak signal in a light sub, say a very faint galaxy. You might not see the arms of the galaxy in an individual sub but once stacked or average with many subs the arms begin to show. Your bands in the darks are the weak signal areas but since they are not random the more you stack the more they show in the simplest of terms.


Mike

[Ryderscope]

Rodney

Think of this like a weak signal in a light sub, say a very faint galaxy. You might not see the arms of the galaxy in an individual sub but once stacked or average with many subs the arms begin to show. Your bands in the darks are the weak signal areas but since they are not random the more you stack the more they show in the simplest of terms.


Mike

Thanks Mike, that explanation works. In fact, when scanning across the image using the readout preview I note that there is only a very small difference in the pixel values. The boosted stretch is making it look worse than it is.

Thanks again for taking the time to look at this.

Rodney

[Stu]

This is a known error with QSI cameras that is related to some internal clocking error with temperature differences.  Or some handwavy explanation like that.  The error is not related to PixInsight.  QSI fixed it. 

There is a two step solution to it which that thread you pointed to ultimately gets to.  If you have trouble finding it call or email QSI and they will help you.  The same thing happened to my camera and after doing the required updates the problem disappeared. 

[Ryderscope]

This is a known error with QSI cameras that is related to some internal clocking error with temperature differences.  Or some handwavy explanation like that.  The error is not related to PixInsight.  QSI fixed it. 

There is a two step solution to it which that thread you pointed to ultimately gets to.  If you have trouble finding it call or email QSI and they will help you.  The same thing happened to my camera and after doing the required updates the problem disappeared.

Thanks Stu, yes I did eventually find reference to the fix towards the end of the CN thread as you mentioned. I have received the update instructions from QSI with the associated software files. As you say, not a Pixinsight issue but I have learnt a bit more about the Image Integration process in the mean time.

[Stu]

There is a Pixinsight "fix" if you want to do some surgery on the banded images.  One thing that I found kind of worked when I had the problem was to rotate the images 90 degrees and then apply the Canon banding reduction script.  It wasn't perfect, but it sort of worked. 

[Ryderscope]

Very clever, worth a try. Interestingly, I only recently noticed the issue with my camera when I ran through some very short, 30 second, subs on NGC104. In this case the bands found their way though to to light masters. Usually I take 5 to 10 minute subs for LRGB and in these cases the bands are not noticeable in the light masters. They are present in the master darks though and I just wasn't aware of them. In the case of NGC104 with my short subs I did manage to make most of the bands go away by re running the Image Integration script on the registered light frames and adjusting the pixel rejection metric. Given that I don't normally notice the banding and I can work around it when I do, I am leaning towards not implementing the fix from QSI.  Sometimes we open other cans of worms when we venture down these paths.

Rodney