Author Topic: CCDBand-Aid repairs vertical bars in Kodak KAI-11000M images  (Read 9547 times)

Offline georg.viehoever

  • PTeam Member
  • PixInsight Jedi Master
  • ******
  • Posts: 2108
    • View Profile
Re: CCDBand-Aid repairs vertical bars in Kodak KAI-11000M images
« Reply #15 on: 2013 September 19 22:49:02 »
...http://bf-astro.com/3_Lum_subEx.zip
This zip file has three calibrated sub exposures. As scaled from the camera, the amplitude of the bars is about 1.20 ADU
http://bf-astro.com/32_mean_combine.zip
This file is a mean combine of 32 unregistered sub exposures. Click STF AutoStretch and you will clearly see the vertical bars.
Wow, this is a really subtle effect. In the 3_Lum_subEx frames it is almost impossible to see. Using a pixelmath formula like "iif($T>0.3842,1,0)" lets you "feel" that it is there, but when you look too closely, you see only noise. In the stacked 32_mean_combine it is clearly visible....

...In the case where the S/N decreases, CCDBand-Aid's result may have been an overcorrection. ....
Any idea why CCDBand-Aid will overcorrect?
Georg
Georg (6 inch Newton, unmodified Canon EOS40D+80D, unguided EQ5 mount)

Offline bob_franke

  • Newcomer
  • Posts: 36
    • View Profile
    • Focal Pointe Observatory
Re: CCDBand-Aid repairs vertical bars in Kodak KAI-11000M images
« Reply #16 on: 2013 September 20 06:03:32 »
Wow, this is a really subtle effect. In the 3_Lum_subEx frames it is almost impossible to see. Using a pixelmath formula like "iif($T>0.3842,1,0)" lets you "feel" that it is there, but when you look too closely, you see only noise. In the stacked 32_mean_combine it is clearly visible....
Yes, it is very subtle. When stretching for magnitude 23 galaxies, the bars become visible... Even with dithered sub exposures.

Any idea why CCDBand-Aid will overcorrect?
CCDBand-Aid calculates the average amplitude for the bar pattern. It is likely that the correction accuracy will vary slightly with different parts of the image. I did not check the overall S/N for the entire image.  With these differences, the bars were still no longer visible in the stacked images with both methods.

Cheers,
-bob

Offline georg.viehoever

  • PTeam Member
  • PixInsight Jedi Master
  • ******
  • Posts: 2108
    • View Profile
Re: CCDBand-Aid repairs vertical bars in Kodak KAI-11000M images
« Reply #17 on: 2013 September 21 15:24:41 »
Sorry Georg, it is definitely a squarewave. The bars are stronger with colder temperatures and longer exposures. The pattern is an odd beast. It does not show up in darks or flats. As I stated in my documentation, it is easiest to see in images of sparsely populated star fields.
...
Hi Bob,

are you sure that it is a square wave? Looking closer at the statistics of stacked 32_mean_combine, I have the impression that it might be a triangle-but even in this stack there is too much noise to be sure. Did you ever try to stack many many many images to get a definitive idea about it?

Also, I wonder why flats don't exhibit the problem. If its really chip timing related, it should be there in any image that has any signal.

Georg
Georg (6 inch Newton, unmodified Canon EOS40D+80D, unguided EQ5 mount)

Offline pfile

  • PTeam Member
  • PixInsight Jedi Council Member
  • *******
  • Posts: 3315
    • View Profile
Re: CCDBand-Aid repairs vertical bars in Kodak KAI-11000M images
« Reply #18 on: 2013 September 21 18:05:03 »
i think the tech note says that if you overexpose a flat you will see the pattern, especially near the top edge of the frame.

Offline bob_franke

  • Newcomer
  • Posts: 36
    • View Profile
    • Focal Pointe Observatory
Re: CCDBand-Aid repairs vertical bars in Kodak KAI-11000M images
« Reply #19 on: 2013 September 21 21:36:02 »
I have never analyzed the pattern other than determining the phase and amplitude.

Matt Longmire, of SBIG, says, "it is a square wave of unknown height and phase."

I just combined 60 flats and the bars were not visible. Also, both CCDOPS and CCDBand-Aid were unable to correctly determine the phase. This is usually an indication of no bars. Applying the CCDOPS fix, with the incorrect phase, added bars to the image.

I do not know why the bars don't appear in flats. My guess is that images, with very short exposures, hide the bars with noise. Maybe stacking two or 300 flats will make them visible.  But, I suspect this will be well beyond the point of diminishing returns.

Cheers,
Bob

Offline bob_franke

  • Newcomer
  • Posts: 36
    • View Profile
    • Focal Pointe Observatory
Re: CCDBand-Aid repairs vertical bars in Kodak KAI-11000M images
« Reply #20 on: 2013 September 21 23:32:15 »
Sorry, of course high signal flats are not noisy.  I have now idea why the bars are not visible in flats. <g>

-bob
« Last Edit: 2013 September 22 19:32:55 by bob_franke »

Offline Carlos Milovic

  • PTeam Member
  • PixInsight Jedi Master
  • ******
  • Posts: 2164
  • Join the dark side... we have cookies
    • View Profile
    • http://www.astrophoto.cl
Re: CCDBand-Aid repairs vertical bars in Kodak KAI-11000M images
« Reply #21 on: 2013 September 23 02:42:35 »
Hi guys

Just to let you know that I'm getting into this "challenge" as soon as I can... Time for development has being scarse latelly though :(

Regards,

Carlos Milovic F.
--------------------------------
PixInsight Project Developer
http://www.pixinsight.com

Offline Chrishet

  • Newcomer
  • Posts: 1
    • View Profile
Re: CCDBand-Aid repairs vertical bars in Kodak KAI-11000M images
« Reply #22 on: 2013 October 26 23:02:13 »
Hi guys

Just to let you know that I'm getting into this "challenge" as soon as I can... Time for development has being scarse latelly though :(

Awesome!  I could certainly use this for my new STXL-11002....what you need Carlos is a larger cup of coffee!  :laugh:

Offline Martin Farmer

  • Newcomer
  • Posts: 29
    • View Profile
Re: CCDBand-Aid repairs vertical bars in Kodak KAI-11000M images
« Reply #23 on: 2014 January 23 20:52:21 »
Hello Guys,

I have been searching for something to help with my banding on the SBIG STL camera I use and came across this thread.
Just wondering if any progress has been made since the last post in October 2013.

Thanks
Martin

Offline Carlos Milovic

  • PTeam Member
  • PixInsight Jedi Master
  • ******
  • Posts: 2164
  • Join the dark side... we have cookies
    • View Profile
    • http://www.astrophoto.cl
Re: CCDBand-Aid repairs vertical bars in Kodak KAI-11000M images
« Reply #24 on: 2014 January 23 20:56:03 »
Thanks Martin from bringing this back. I'm currently working on another new noise reduction tool. I'll work on this next week.
Regards,

Carlos Milovic F.
--------------------------------
PixInsight Project Developer
http://www.pixinsight.com

Offline Martin Farmer

  • Newcomer
  • Posts: 29
    • View Profile
Re: CCDBand-Aid repairs vertical bars in Kodak KAI-11000M images
« Reply #25 on: 2014 January 23 22:31:05 »
Thank You Carlos,

That's good news indeed.
I only noticed the banding after zooming into my images with Pixinsight, and once you know they are there it drives you mad.
I and indeed others look forward to your solutions.

Many thanks
Martin

Offline Carlos Milovic

  • PTeam Member
  • PixInsight Jedi Master
  • ******
  • Posts: 2164
  • Join the dark side... we have cookies
    • View Profile
    • http://www.astrophoto.cl
Re: CCDBand-Aid repairs vertical bars in Kodak KAI-11000M images
« Reply #26 on: 2014 February 05 22:51:16 »
Could anybody share some images again, please? I'll begin with this tomorrow, so I'll need to inspect the fourier spectrum of a few samples.

So far, I think that I have two paths to follow:
- A notch filter. Automatically find the frequencies that are giving problems, and fix them. I'll take a look at some optimized/adaptive algorithms.
- A TGV restoration in the fourier domain. I've being playing with TGV to solve undersampled reconstruction in the fourier domain for my researchs at the university. I may be able to modify the code of the soon to be published TGVRestoration to handle the banding problem, but since the module still needs some work, it may take some time.

Regards,

Carlos Milovic F.
--------------------------------
PixInsight Project Developer
http://www.pixinsight.com

Offline bob_franke

  • Newcomer
  • Posts: 36
    • View Profile
    • Focal Pointe Observatory
Re: CCDBand-Aid repairs vertical bars in Kodak KAI-11000M images
« Reply #27 on: 2014 February 25 00:12:52 »
Hello Carlos,

Sorry I did not notice your post earlier. The files, I posted earlier, did not make it to the new provider for my website.

Here are the images for development purposes.

http://www.adrive.com/public/TGjfcR/3_Lum_subEx.zip
This zip file has three calibrated sub exposures. As scaled from the camera, the amplitude of the bars is about 1.20 ADU

http://www.adrive.com/public/PWArfa/32_mean_combine.zip
This file is a mean combine of 32 unregistered sub exposures. Click STF AutoStretch and you will clearly see the vertical bars.

Regards,
Bob

Offline Carlos Milovic

  • PTeam Member
  • PixInsight Jedi Master
  • ******
  • Posts: 2164
  • Join the dark side... we have cookies
    • View Profile
    • http://www.astrophoto.cl
Re: CCDBand-Aid repairs vertical bars in Kodak KAI-11000M images
« Reply #28 on: 2014 February 25 03:11:19 »
Thanks Bob. I'm comming back from my vacations next week.
I already began to write a new module. It worked pretty well with high frequency noise (vertical banding of only a couple of pixels width), but it yielded poor results with low frequency one. I'm going to try a different approach after I finish an implementation of an optimal notch filter for another process.

I think that i'm going to write a solution pretty close to the canon banding script. So, if you have any suggestions on that field, I'm all ears.
Regards,

Carlos Milovic F.
--------------------------------
PixInsight Project Developer
http://www.pixinsight.com

Offline bob_franke

  • Newcomer
  • Posts: 36
    • View Profile
    • Focal Pointe Observatory
Re: CCDBand-Aid repairs vertical bars in Kodak KAI-11000M images
« Reply #29 on: 2014 February 25 07:01:17 »
Sorry that I cannot offer much help.

If you are developing specifically for the KAI-11000M sensor, you can assume that the unbinned period is 32 and the pulse width is 16. With 2x2 binning, the period is 16 with a pulse of 8.

You just have to figure out the phase and amplitude.

Good luck,
Bob