Vertical Banding

[Paul Mahoney]

Hi, Last night I was trying Maxpoint but with the moon so bright I couldn't auto calibrate the mount so I decided to give up but finished by taking 50 subs of the moon so that I could try to use PI to debayer it. The subs were taken in Maxim DL and calibrated as well. When I debayered the resulting image in PI I was left with faint (red) banding running top to bottom across the image. I tried the same thing in Nebulosity and got the same banding. In Maxim after debayering  there was no sign of the banding. I have attached the finished PI image in case anyone can throw some light on why it should happen and what I can do to avoid it.

I would be grateful for any input.

Kind regards,

Paul

London, UK

[Carlos Milovic]

Looks like a perfect example for a fourier based noise reduction

I don't know the cause of it... seems pretty regular, so it may be some kind of electrical interference. A workaround may be to decrease a lot the color saturation.

[Nocturnal]

Hi,

the debayer mechanism in PI isn't very advanced, it's either super pixel or bilinear. The good thing is that it also won't insert any artifacts. This means I am quite sure that your original data contains this pattern and the debayering makes it visible. It's possible that Maxim does debayering a little differently and can somehow suppress the banding but more likely the display of the data wasn't quite the same. Try stretching the Maxim image or increase the saturation and see if the bands appear.

[Paul Mahoney]

Hi Carlos and Sander,

Thanks for your replies. Reducing the colour saturation works ok, I am just a bit bemused as to why it has happened. Perhaps it is always there but not noticeable in normal dark sky objects. Sander I tried what you said in Maxim but cannot recreate the banding whatever I do.

So, something I will have to live with, for the time being anyway.

Again, many thanks and kind regards,

Paul
London, UK

[Nocturnal]

Hi Paul,

I wrote the debayer module so I'm certainly interested in this. If you're willing to let me pursue this a bit more, could you upload a single raw frame and that same frame debayered by Maxim? You could crop the raw frame before debayering to reduce the file size, as long as it still shows the banding in PI.

http://deepskystacker.wikispaces.com/Where+do+I+upload+large+files%3F

[Paul Mahoney]

Hi Sander,

I hope I haven't misled you but I am using Niall's CMYG batch debayering script. 

Let me know if you still want to see the files.

Kind regards,

Paul
London, UK

[Nocturnal]

Hi Paul,

ah! Now I feel better I imagine Niall might be interested in the data though

[Paul Mahoney]

Hello again,

Whilst waiting for it to get dark, I decided to run some more stacking examples in Maxim. I had been using the average setting but this time tried the other settings as well. 'Lo and behold'  the banding appears, really pronounced. I went back and tried the average setting again and if I used the stretch window to make the darker areas almost black, I could see the banding, quite feint, but definately there.

So, I am back at the beginning and Carlos suggestion that it may be electricl interference could hold true. I will check my cable runs etc., next time to see if it makes a difference.

Thanks again for your imput.

Paul
London, UK

[Niall Saunders]

I hope I haven't misled you but I am using Niall's CMYG batch debayering script

Hi Paul,

In fact, I hope it isn't me that has misled you!!

My deBayer routine was specifically written for CCD cameras that use a SECONDARY colour filter array, such as used in the early Meade DSI range of cameras.

These imagers relied on a Sony CCD chip that has a CMYG colour filter array as opposed to the RGGB (or RGBG) colour filter array now more popular in modern CCD imagers (DSLRs included). What is more the two CFAs are also totally different in size - that of the DSI style is actually based on a repeating array of 2 rows by 4 columns, whereas the more common array is now a 2x2 array.

However, my algorithm places NO RESTRICTION on what it will 'deBayer' - it just takes the raw data 'in good faith' and blindly applies the algorithm. But, if the source data was NOT the raw data from a CCD with 2x4 CMYG CFA, then all bets are off as far as any result is concerned

So, if you are NOT using a CMYG-array colour camera, then you should consider the alternative deBayer routines that have been written since I published mine. In reality, my routine really has a VERY limited scope of application. Very, very few CCD imagers were ever manufactured using the CMYG array. Consequently, there were very few deBayer algorithms that I could find, and NONE of them seemed to work worth a damn (at the time) - or, at least, I certainly couldn't make them work So, I grabbed the bull by the horns and wrote my own - using the incredible power of PI's PJSR platform to achieve this. When Juan had to help me solve a couple of problems, he then asked me if my script could be included with all future releases of PI, and I was only too happy for this to happen.

Why was it included in the standard PI distribution, if its eventual use was going to be so limited? Well, because I had had to 'force' myself to understand how PJSR scripts were put together, I drenched my script with hundreds of comments, as a means of reminding me of how the heck my script was supposed to work - and Juan realised that my liberal sprinkling of comments might well be useful to others who might also want to pick up PJSR and give it a go.

So, sorry if my script has given you a whole extra load of grief as it blindly deBayered your non-CMYG data. But, hey, I am glad you at least played with it!!!

[Paul Mahoney]

Hi Niall and thanks for your message.

I was using an Orion SSDSI II ccd which has the cmyg colour filter array. NOTHING wrong with your script, in fact it is superb and I'm very grateful that you put in the time and effort to produce it.

My issues seem to have reverted back to Maxim DL as I can now reproduce the problem there before it ever gets to PI.

I recently upgraded Maxim, so I am wondering if that has something to do with it, partricularly since I can now get banding on images taken months ago that never had banding and reprocessed in the latest Maxim version. I have a technical support ticket out with them at the moment.

Once again, thanks for your input and kind regards,

Paul
London, UK

[Niall Saunders]

Hi Paul,

Well, I am delighted that all my effort has been worthwhile - in fact, you may even be the ONLY user of the script now that I am spending more time imaging with my DSI Pro, and now that my neighbour is using the DSI IIIC in his observatory (which now uses the more common RGBG CFA).

What would be useful is if you could let me know the settings that you are typically using to deBayer the Orion RAW images - I could then update the script and let Juan have a copy for future releases.

Glad to have been of help.

[Paul Mahoney]

Hi Niall,

This all started because I wanted to get as much processing done in PI as possible. It is a hard program to learn but I have decided to stick with it and try (with the aid of all you helpful people) to become proficient using it.

I had just started using your cmyg debayer script when I discovered my banding problem. In fact it was your script that exposed the problem to me for the first time. I was using the Nebulosity default settings but of course with the banding I am not in a position to say whether these settings are suitable or not. I can calibrate and colour convert in Maxim and get a result that doesn't show the banding, regardless of how it is stretched yet if I do the same thing without colour conversion the banding sticks out like the proverbial 'sore thumb'.

I have been in touch with Maxim tech support and they say that the problem is with the camera so I have now raised a ticket with Orion to see what they say.

If it turns out that the camera is U/S, God knows how I will get 'her indoors' to agree to me buying another one  However, I will wait and see what Orion have to say about it.

Kind regards,

Paul
London, UK

[Carlos Milovic]

Hi Paul

First thing to try is using other capture software, and see if you get that banding. Inspect each channel, and you'll see some bands crossing horizontally too, inside the vertical bands.

[Niall Saunders]

Hi Paul,

I assume that you have tried all the different 'options' in my Batch deBayer Script? It has been a while since I sued it, but I do remember all sorts of 'nasties' if the fundamental setup was not 'just right'.

You might want to look at the optiona available when you press the <deBayer Setup> button from the main GUI:-


In the first section, DeBayer Parameters, there is nothing much to be gained by changing the colour-conversion array from "Theorectical_1" to any other method. In the new versions of PI, the BackgroundNeutralisation and ColourCorrection processes are now powerful enough to void any attempts at colour correction 'during' the DeBayer process.

The same argument can be applied to the third section as well - especially the colour correction curve (which, in any case, requires that you have created an appropriate curve for YOUR camera). I would therefore recommend that you un-tick ALL the boxes in this section whilst we try to identify any other problems - this is now actually very much the best way forward, as ImageIntegrtaion within PixInsight is FAR, FAR, FAR more powerful than my trivial attempts at 'intensity matching' within the script.

Which just leaves the 'middle section' of this part of the GUI.

Here I am giving you the option of determining 'where' on your CCD pixel-grid the actual CFA 'starts'. And, there are EIGHT possible start points - one for each location on a two-column by four-row CFA array. Now, this was the 'key' to my deBayer approach - as, from what I could tell at the time, no other package was providing this level of flexibility. I now know that Nebulosity 'does', but - if this was available at the time I needed it - it was buried in an illogical menu location that I was not able to find (and remains there even to this day!!!).

What I recommend is that you, somehow, obtain a DAYLIGHT exposure of a bright and colourful still-life scene. Yes, I know that British Winter Time is upon us already - so 'bright and colourful' doesn't come easy - but, be inventive. If you can't, then I have provided a TESTCARD that I used elsewhere on this Forum (have a look at http://pixinsight.com/forum/index.php?topic=1997.0) which you can try and image, providing that you can find the best print size to suit whatever lens you can attach to your camera.

Then, having acquired a RAW frame of such a scene (or the test card) you should try deBayering the image using ALL EIGHT possible options - and decide which of these gives you the 'closest' result. Four of the options should be noticeably 'wrong' - there will be LOADS of horrible banding. Three of the remaining options will be better than the last one - but only one of the three options will really 'pop'.

You can ignore the X-flop and Y-flip options as well - I found that applying either, or both, of these options simply provided an identical 'group' of eight possible options as described above - only in a different 'order',

Once you have determined, empirically, which CFA-offset option seemed to be 'best', you can try applying this offset to a RAW astronomical image. You still will not have 'perfect colours' though - remember, BackgroundNeutralisation and ColourCorrection will sort that out for you later on.

Then, please let me know what X and Y offset values worked best for you, so that I can include them in the PJSR code for the future, so that any other users may be able to benefit. You could also try the same process (and X/Y offset values) in Nebulosity as well - I can't remember the convoluted menu path to get to these options, but I do remember it was something 'like' >File >Preferences, or some such.

Let us know if this helps.

[Paul Mahoney]

Firstly, Hi to Carlos and thanks for the tip to try different software. Using Nebulosity I took some daylight shots as the weather here is pretty awful and likely to be for a while now according the forecasts. I stretched and manipulated the images in all kinds of ways but couldn't reproduce the banding. So I am back in touch with Maxim to get their views now. If I align the 'Neb' images in Maxim, the banding immediately appears. If I ever get to the bottom of this I will let you know the outcome.

Secondly, Hi to Niall and thank you too for a detailed insight into your script. This clarifies a couple of points that I was not too sure about.

You will gather from my reply to Carlos above that it may be some time before I get to use PI as I don't feel enclined to proceed until I can be confident of the quality of the Raw images I can get from my camera. Once I am back, up and running, and find the best settings for my camera in your script, I will definately let you know.

Thanks to you both again and kind regards,

Paul
London, UK